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I .  THE  TASK 

Research  conducted  under  Phase  I  of  Project  ECHO  has  sought  to 
develop  a  methodology  for  understanding  alien  cultures  and  communicating 
more  effectively  with  these  cultures.  The  former  objective,  understanding, 
accrues  from  the  method's  ability  to: 

1.  identify  subcultures  within  a  population 

2.  identify  the  value  systems  of  these  cultures 

3.  determine  the  degree  of  internalization  of  specific  values 
of  members  of  the  subculture 

•  quantify  differences  in  social  perceptions 

5.  identify  the  "power  structure"  operating  in  the  culture 

The  ECHO  technique  utilizes  a  "projective  survey"  format;  the  method 
is  projective  because  the  subjects  (Ss)  are  permitted  to  define  some 
variables  which  are  usually  predetermined  by  the  investigators,  and  a 
survey  because  groups  rather  than  individuals  arc  the  object  of  concern. 

The  value  of  this  type  of  attitude  survey  and  the  theoretical  logic 
underlying  its  development  ate  explicated  below. 
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XI,  THEORETICAL  CONSTRUCTS  AND  ASSUMPTIONS 

Current  theories  hold  that  attitudes,  which  ECHO  attempts  to  assess, 
have  three  components:  cognitive  (with  focus  primarily  on  the  evaluative 
part),  affective,  and  behavioral.  People  tend  to  seek  consonance  among 
the  three  components,  so  that  a  given  attitude  has  internal  consistency: 
if  the  evaluation  of  an  object  is  positive,  then  the  feelings  .bout  and 
behavior  toward  that  object  are  likely  also  to  be  positive,  h  person 
who  evaluates  an  object,  person,  or  concept  positively  will  be  predisposed 
to  behave  positively  toward  it;  conversely,  a  person  is  likely  to  act  in 
opposition  to  an  object,  person,  or  concept  that  he  evaluates  negatively. 

If,  on  the  other  hand,  the  system  is  out  of  balance  (e.g.,  feelings 
and  evaluation  are  positive  while  beha-lor  is  negative),  the  individual 
has  a  strong  tendency  to  bring  it  into  balance  by  modifying  either  the 
behavior  or  the  feelings  and  evaluation. 

Values  can  be  considered  to  be  enduring  systems  of  positive  or 
negative  evaluations.  Thus,  if  a  value  system  is  understood,  correspond¬ 
ing  behaviors  can  be  estimated;  correspondingly,  behavior  can  be  influenced 
by  modifying  values.  The  primary  ECHO  task  is  to  discover  value  systems 
by  a  method  that  is  analogous  to  sur  ’ey  sampling  (polling) . 

With  the  recent  advances  in  polling  techniques,  very  small  samples 
of  data  can  be  used  with  great  confidence  to  predict  public  opinion  on 
a  wide  range  of  issues.  However,  polls  are  only  applicable  to  populations 
about  which  much  is  already  known.  Asking  the  right  question  of  the 
right  people  is  difficult  even  in  a  culture  with  which  we  are  intimately 
familiar.  The  ECHO  method  obviates  tiis  "previous  knowledge"  problem 
by  reversing  the  polling  process.  Pc  Us  ask  the  respondent  to  assign  an 
evaluation  to  a  preselected  topic;  the  ECHO  technique  ass'gns  an  evaluation 
and  asks  the  respondent  to  think  of  a  behavior  which  carries  this 
evaluation. 
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A  specific  example  of  this  kind  of  projective  question  is:  "What  is 
a  good  thing  that  you  could  do  which  someone  would  praise  you  for  doing? 
Who  is  the  someone  who  would  approve?"  The  format  of  the  question  can 
be  modified  by  varying  the  assigned  evaluation  and  "role".  The  role  in 
this  example  was  "you".  Another  role  might  be  "you  as  a  nurse"  (student, 
employee,  etc.).  Questions  can  be  cast  in  several  forms  according  tc  the 
needs  of  rhe  investigators. 

Answer?  to  the  query  "Who  would  approve  (disapprove ) ?"  provide 
information  about  the  subjects'  perceived  "powerstructure" ,  i.e.,  the 
sources  of  positive  and  negative  reinforcement  which  control  his  behavior. 

The  projective  survey  technique  was  conceived  by  Professor  Alex 

1  2  3 

Bavelas  and  applied  in  a  variety  of  settings  by  Kalhorn,  Warner, 

4  5 

Havinghurst  and  Neugarten,  and  Rice.  Research  conducted  under  Project 
ECHO  has  differed  from  previous  studies  in  a  variety  of  ways.  The  current 
research,  for  example,  uses  indigenous  classifiers  rather  than  "expertt" 
to  categorize  the  raw  inputs,  and  validity  tests  have  been  introduced. 

New  forms  of  the  instrument  and  methods  of  statistical  analysis  have 
also  been  developed  during  this  period. 


UNCLASSIFIED 


3 


DAT  A  C 


UNCLASSIrlED 


tV3l-RV 


UNCLASSIFIED 


Figure  1.  The  ECHO  Methodology 
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III.  THE  ECHO  METHODOLOGY 

The  ECHO  methodology,  as  developed  by  General  Research,  is  a  nine- 
step  process  Irom  data  collection  to  production  of  valid  descriptions 
of  value  hierarchies  and  power  structures  in  the  test  population. 

Figure  1  represents  the  process  diagramatically . 

A  brief  summary  of  the  significant  elements  in  the  ECHO  methodology 
follows: 

1.  The  problem  is  tentatively  defined  and  an  appri  'riate  form 
of  two  projective  ECHO  questions  is  selected. 

2.  A  population  is  identified  and  a  sample  selected. 

3.  Subjects  are  asked  to  generate  ten  anonymous  answers  to  each 
of  the  ECHO  questions.  Answers  are  recorded  on  preprinted 
and  coded  IBM  tabulating  cards. 

4.  The  response  cards  are  divided  into  logical  groups  (male- 
female,  good-bad,  etc.)  and  each  group  of  data  is  categorized 
by  three  different  teams  of  classifiers  (£s) .  Cs  divide  the 
cards  into  categories  and  provide  brief  descriptions  of  each 
category's  contents.  They  then  rank  the  categories  on  some 
assigned  dimension  of  importance. 

5.  The  classified  cards  are  punched  with  the  two-^igit  codes 
which  identify  the  classifying  team  and  the  category  descrip¬ 
tion  which  was  assigned.  Coded  cards  are  submitted  to  a  five- 
step  computer  analysis  which  produces  value  system  and  power 
structure  outputs. 

6.  Equal-length  lists  of  "values"  are  extracted  by  selecting  the 
titles  of  the  five  categories  with  the  highest  frequencies  (f) 
(i.e.,  the  largest  number  of  cards)  in  each  classification 
system. 

7.  The  lists  are  presented  in  a  forced-choice,  paired-comparison 
format  to  a  second  sample  of  subjects  from  the  same  population. 
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Each  list  is  paired  with  one  of  the  others,  or  with  a  list  from 
some  other  population;  Ss  are  asked  to  select  the  set  of  values 
in  each  pair  which  is  "most  important  to  them".  The  hypothesis 
is  that  the  Ss  will  "elect  (significantly  more  than  chance)  the 
list  which  came  from  their  own  population  and  will  reje  t 
(significantly  mors  than  chance)  lists  which  did  not  come 
from  their  population. 

8,  Those  lists  which  were  selected  as  hypothesized  are  assumed 
to  come  from  valid  representations  of  a  portion  cf  the 
population's  value  hierarchy. 

9.  Concurrently  with  the  above,  otner  data  are  analyzed  and  the 
results  combined  with  (8). 

The  above  apercu  constitutes  only  ^  cursory  outline  of  the  ECHO 
methodology.  A  detailed  description  and  justification  of  each  variable 
in  the  process  (20  cards,  3  classifications,  5  most  frequently  mentioned 
categories,  etc.)  is  given  in  Secs.  V  through  VII. 

The  methodology  described  above  is  the  product  of  a  large  number 
of  individual  experiments.  The  contribution  that  each  experiment  made 
to  a  particular  element  of  the  methodology  is  described  below;  the 
chronological  sequence  of  events,  hypotheses,  and  experiments,  are  reported 
in  Appendix  I. 
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IV.  SUBJECT  POPULATIONS  USED  IN  PHASE  I 

Eleven  different  populations  participated  in  the  Phase  I  research. 
These  groups  included:  freshmen,  seniors,  and  graduate  students  from 
three  different  universities  (Northwestern,  UCLA,  and  Stanford);  clerical 
workers  from  two  Southern  California  companies;  student  nurses  from  a 
Cv icago  hospital;  two  classes  of  elementary  school  children  (third  and 
sixth  grades);  Mexica.i-American  residents  of  "East  L.A.";  and  two  groups 
of  Cuban  exiles  who  now  reside  in  Soutnern  California.  In  many  cases, 
multiple  samples  were  drawn  from  larger  populations  and  a  different  experi¬ 
mental  treatment  was  applied  to  each  sample  group.  A  summary  of  the 
populations  used  in  research  completed  under  this  contract  is  contained 
in  Table  1.  This  table,  however,  does  not  reflect  the  actual  number  of 
samples  tested,  experimental  treatments  applied,  or  subcultures  studied 
(a  single  population  may  have  several  subcultures  operating  simultaneously). 
For  convenience  of  presentation  this  detailed  information  has  been 
included  in  Appendix  I. 


TABLE  1 

SUMMARY  OF  SUBJECT  POPULATIONS  AND  EXPERIMENTAL  TREATMENTS  USED  IN  PROJECT 

ECHO,  THASE  I 


Population 

Code 

Number  of. 

Ss  in 

Population 

Number  of 

Classifications 

Completed 

Message 

Session 

Held? 

No. 

Ss 

Number 

of 

Forms 

UCLA 

B 

137 

17 

Yes 

70 

(2) 

Northwestern 

T 

68 

11 

Yes 

83 

(6) 

Stanford 

A 

72 

2 

No 

Nurses 

N 

52 

12 

Yes 

37 

(2) 

Carnation 

Company 

C 

54 

2 

Yes 

31 

(2) 

Prudential 

Company 

P 

93 

3 

Yes 

98 

(2) 

Third  Graders 

3 

17 

3 

No 

Sixth  Graders 

6 

24 

3 

No 

Cuban  Exiles 

CE/K 

25/18 

10/8 

Yes 

25 

(2) 

Total 

560 

71 

387 

16 
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V.  DATA  COLLECTION 

The  ECHO  question  can  be  cast  in  a  variety  of  forms  according  to 
the  needs  of  the  investigators;  different  forms  tap  different  attitudes. 
There  are  five  major  variables  in  the  question: 

1.  Role  assignment  (p) :  the  socially  prescribed  position  to  be 
used  as  the  frame  of  reference  for  answering  the  question 

2.  Event  (c) :  the  kind  of  occurrence,  usually  a  behavior, 
solicited 

3.  Event  evaluation  (n) :  the  positive  or  negative  quality  of 
the  event  (behavior) 

4.  Reinforcement  (tt)  :  specification  of  either  positive  or 
negative  reinforcement 

5.  Source  (o) :  che  person(s)  or  concept(s)  which  provides  the 
reinforcement . 

In  some  cases  a  sixth  variable  may  be  introduced  to  define  the 
relationship  between  the  role  holder  (p)  and  another  role,  role  holder, 
or  group. 

For  example,  a  question  designed  to  tap  the  areas  of  conflict 
between  segments  of  a  society  might  take  the  following  form: 

What  is  something  a  person  like  you  (p)  could  do  (e)  that 

your  friends  would  say  was  foolish  (ri)  but  that  someone  else 

would  praise  (n)  you  for  doing?  Who  would  that  someone  else 

(o)  be? 

Two  forms  of  the  projective  question  have  been  used  to  collect  data 
during  the  Phase  I  research:  "What  is  a  (good/Lad)  thing  to  do?"  and 
"What  as  a  (good/bad)  thing  that  could  happen?"  The  latter  question  was 
used  during  the  summer  1966  pilot  scudies  and  has  been  given  only  cursory 
consideration  during  this  phase  of  the  research. 


8 


UNCLASSIFIED 


UNCLASSIFIED 


Subjects  were  asked  to  write  their  answers  on  IBM  cards,  which 
became  the  basic  data  for  all  subsequent  operations. 
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VI.  CLASSIFICATION 

The  classification  process  is  an  integral  part  of  the  ECHO  methodology. 
It  is  based  on  the  hypothesis  that  indigenous  classifiers  (members  of  the 
exact  population  under  study)  are  able  to  make  fine  discriminations  among 
statements  from  their  own  subculture  and  to  understand  the  nuances  of  the 
language,  and  that  they  do  this  better  than  "experLs".  This  hypothesis 
was  supported  in  the  multiple  classification  studies,  reported  below,  and 
in  the  validity  test  sessions  (Sec.  VII). 

A.  METHOD 

The  classifiers  worked  first  individually,  each  with  one-third  of 
the  deck,  sorting  the  cards  into  categories  that  had  meaning  to  them.  The 
investigator  did  not  give  examples  of  categories  since  any  example  would 
tend  to  structure  the  process  in  terms  of  the  investigator's  frame  of 
reference;  ECHO  is  interested  in  discovering  the  frames  of  reference  of 
the  subject  population. 

Although  classifiers  take  varying  amounts  of  time  for  this  step, 
the  modal  time  is  approximately  30  min.  After  all  three  had  completed 
the  individual  sorting,  they  joined  together  to  develop  a  single  set  of 
categories  to  include  all  cards.  One  person  read  the  cards  in  one  of  his 
stacks  and  the  others  added  cards  that  meant  the  same  thing.  The  process 
was  continued,  with  the  other  two  taking  turns  reading  cards,  until  all 
the  cards  had  been  placed  into  categories.  They  then  titled  the  categories, 
either  making  up  titles  or  using  a  representative  answer  found  on  one  of 
the  cards  in  that  category. 

The  investigator  assigned  a  two-digit  code  number  to  each  category, 
the  cards  were  punched,  and  then  they  were  classified  by  another  team. 

B.  QUANTITY  (n  DATA 

A  team  can  comfortably  classify  the  data  cards  of  25-30  subjects, 
that  is,  250-300  cards. 
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This  conclusion  was  reached  by  a  process  of  successive  approximations. 
The  first  classification  session  at  UCLA  was  designed  to  test  the  process 
as  well  as  to  generate  data.  Two  teams  each  classified  approximately  1400 
cards.  Two  sessions,  six  hours  and  five  hours  long,  were  required.  The 
high  motivation  and  enthusiasm  that  seemed  clear  at  the  beginning  changed 
to  discomfort,  fatigue,  boredom,  and  frustration  as  the  hours  passed. 

In  the  studies  at  two  Southern  California  companies,  the  classifica¬ 
tion  teams  worked  with  approximately  500  cards  (516,  526,  599,  600  and 
472  cards).  Although  fatigue  and  boredom  were  not  as  pronounced  as  in 
the  UCLA  sessions,  the  investigators  were  still  impressed  by  the  decreased 
efficiency  as  time  passed. 

In  the  Stanford  study,  the  two  classification  teams  each  worked  with 
700  cards.  One  team  completed  its  task  in  approximately  5  hours;  the 
other  team  stopped  work  at  the  end  of  4-1/2  hours,  and  completed  the  task 
in  3  hours  the  following  morning. 

In  the  Northwestern  study  the  classifiers  worked  with  decks  of 
approximately  325  cards.  The  Nurses  study  classifiers  worked  with 
approximately  250  cards.  In  both  groups,  a  complete  classification 
took  from  2  to  3  hours.  The  team  members  did  not  appear  fatigued;  informal 
interview  data  supported  the  observation.  In  both  studies,  a  second  set 
of  four  teams  reclassified  the  data  two  days  later  under  virtually  Identical 
conditions.  Although  the  second  group  had  had  no  experience  with  the 
process,  the  time  to  complete  the  task  varied  from  1-1/2  to  2-1/J  hours. 

A  possible  explanation  for  this  time  reduction  is  that  the  second  group 
learned  from  classmates  that  the  task  was  nonthreatening  and  therefore 
approached  it  with  more  confidence. 

In  the  course  of  the  UCLA  multiple  classification  session,  described 
below,  11  classification  sessions  with  150  card*  vere  run.  The  modal 
time  was  under  two  hours;  only  one  group  took  3  hours.  Four  sessions 
using  300  cards  were  completed  in  2-1/2  to  3  hours  . 
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The  Cuban  study  classifiers  worked  with  packets  of  90,  100,  110, 
and  140  cards.  During  each  three-hour  session  each  group  classified 
twice. 


C.  MULTIPLE  CLASSIFICATION  OF  DATA 

Current  findings  indicate  that  a  given  deck  of  cards  should  be 
classified  by  three  different  teams  to  give  a  measure  of  reliability. 
Also,  the  categories  from  one  classification  can  be  used  to  interpret 
the  categories  of  another. 


Multiple  classification  appears  to  be  one  of  the  most  powerful 
tools  for  data  analysis.  The  process  is  simple:  two  or  more  teams 
independently  classify  the  same  set  of  cards.  Studies  conducted  with 
UCLA,  Northwestern,  Nurses  and  Cuban  groups  indicated  that  when  classifiers 
came  from  the  same  groups,  reliability  tended  to  be  high  (i.e.,  the  same 
cards  were  grouped  together),  but  these  same  teams  used  different  numbers 
of  categories  and  different  category  widths.  Examination  of  the  structures 
of  the  categories  indicates  that  when  the  cards  placed  in  a  large  category 
by  one  group  were  distributed  into  two  or  more  categories  by  another 
group,  the  category  labels  had  semantic  equivalence. 

Example:  The  category  "Hel  ler  people"  from  UCLA  female 
(FI)  classifiers  was  distributed  by  another  group  (F2)  of 
classifiers  into  three  categories  with  titles  "Help  someone 
at  you.-  own  expense,"  "Help  those  less  fortunate,"  and  "Help 
others."  Since  both  the  language  and  culture  were  familiar, 
it  was  possible  to  state  with  some  assurance  that  the  classi¬ 
fiers  were  in  agreement,  but  that  the  second  group  made 
finer  discriminations. 


When  classifiers  are  drawn  from  somewhat  different  populations  the 
classifications  have  less  overlap,  presumably  the  result  of  a  difference 
in  social  perceptions.  Table  2  shows  male  data  classified  by  a  team  of 
male  (M2)  and  a  team  of  female  (F4)  classifiers.  An  examination  of  the 
titles  indicates  that  they  are  not  in  disagreement  but  only  organized 
differently.  Thus,  the  male  concept  of  "Be  a  better  person  and  relate  to 
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TABLE  2 


ucu  MALE  (M2)  DATA  CLASSIFIED  BY  F4  and  „2 


„r  .  ~~  ^ale_(M2)_CiaSsiflcat<nn 

Live  up  to  self-ideal"  (f  =  7) 

"Be  more  easy-going"  (f  *  4) 

"Have  more  self-confidence"  (f  .  3)  ^  PGrS°n  and  reJ 

"Realize  minor  goals"  (f  =  3)  C°  ^  environment  better"  ( 

"Be  more  liveable"  [sic]  (f  «  3) 

"Improve  sphere  of  knowledge"  (f  -  x) 

my  environment  better"  meant  ^  ^ 

up  to  self-ideal"  "Be  mor  '  656  W°men’  3  number  of  things:  "Live 

"Realize  minor  go  is"  "Be  ^  ^  -lf-confidence", 

guals  •  Be  more  liveable"  fsid  anj  »T 

knowledge".  A  per,on  from  our  culture  can  Jit  ,  ^  ^  °f 

that  particular  breakdown  of  the  latter  ”  ^  “  ‘h‘  logl,:  of 

this,  males  and  females  from  th  68017 '  T”°  gtOI,pS  SUCh  89 

well:  a  number  of  d^»t""  ‘“h001'"6' 

made  by  other  groups  to  see  If  th  CheCke<i  tha  classi£iCacior 

~  o-r^oup:  Ire  ^  ^  ^ 

the  cards  Into  those  categories  The  ’  a"d  aEke'1  to  so 

multiple  classification  method  a  fl8ree'',e”t  *1",0't  ab9°lata  •  Th 
-hod  Itself  IS  u„dergo:::efir;;.POte"tlaUy  **—•  -  - 

“•  m^»OVS  CLASSIFIERS  NECESSARY? 

able  than  experts  to^nlL 'tte  “1'*n0"*  Cla99ifl9ta  <£»>  are  more 
tbls  ability  accrues  from  the  ““g"1“> 

with  the  subject  population.  greater  familiarity 
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Example:  One  group  of  University  Cs  distributed  a  batch  of 
cards,  which  the  exports  (a  professor  and  a  graduate  studen*) 
defined  as  "Be  kind  to  roommates,"  into  two  different  cate¬ 
gories:  "Be  kind  to  dorm  roommates"  ani  "Be  kind  to  apartment 
roommates."  In  explaining  their  actions  the  £s  pointed  out 
that  apartment  roommates  are  secured  through  a  "self-selection" 
process  while  the  dormitory  roommates  ate  assigned  by  an 
impersonal  computer;  therefore,  more  tolerance  and  understanding 
is  due  the  former. 

The  importance  of  indigenous  classifiers  is  evident  when  the  work 
of  alien  Cs  is  compared  witli  the  classif ications  made  by  indigenous  Cs. 

For  example,  a  batch  of  1  1  *5  I'CI.A  cards  was  classified,  at  different  times, 
by  c.  group  of  three  UCLA  coeds  and  a  group  of  three  female  Cubans  .  The 
categories  created  by  each  team  and  tne  cards  composing  each  of  these 
categories  are  shown  in  matrix  form  in  Table  3.  Obviously,  the  UCLA  Cs 
were  able  to  make  finer  discriminations;  the  shaded  area,  for  instance, 
shows  that  the  Cubans  lumped  together  five  UCLA  categories  ("study  hard 
to  get  good  grades";  "prepare  for  future  career";  "graduate  in  alloted 
time;"  "learn  for  learning's  sake;"  and  "learn  new  skills")  into  a  single 
general  category,  "learning  and  get  good  grades."  The  matrix  has  other 
examples  of  this  discrimination  differential.  Experimentation  with  both 
Cuban  and  College  Cs  indicates  that  these  apparent  differences  in  social 
perceptions  are  not  artifacts  of  culture-specific  ways  of  categorization 
(e.g.,  using  few  or  many  categories),  but  represent  different  ways  of 
viewing  the  same  concepts,  which  is  what  one  might  expect  between 
cultures. 


E.  CONCLUSIONS 

1.  The  optimum  number  of  cards  for  one  classification  session  is 
250-300.  (Note  that  investigations,  reported  below,  indicate 
that  this  number  yields  a  stable  data  base.) 

2.  If  larger  numbers  are  to  he  classified,  the  sessions  should 
be  broken  so  that  no  session  is  longer  than  three  hours.  A 
minimum  of  two  hours  should  elapse  between  sessions. 

3.  Each  deck  should  be  classified  three  times. 
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TABLE  3 

EXAMPLE  OF  TWO  CLASSIFICATIONS  OF  SAME  DATA 

"Eyeball  Analysis"  Indicates  That  the  UCLA  Cs  Were  More  Able  to  Make  Finer 
Distinctions  Among  Concepts.  Shaded  Areas  Illustrate  Differences  in  Two 

Teams'  Social  Perceptions. 


f'i 

L. 

O 

/ 

L. 

’-I  vr, 

•O 

-e 

<Si 

ut 

-J 

UCLA  It 

FEMALFS  CLASSIFYING  Fj 

Get  a  Good  .lob 

Cleanings 

Helping  Others 

Family 

To  One  Self 

i  Personal  Interest 

Learning  and  Get  Gooo  Grades 

Participate  with  Other  People 

Learn  More 

Take  More  Classes 

Lose  We<ght 

Love  for  Other  People 

Go  to  Church 

Be  on  Time 

TOTALS 

Propate  for  Future  Career 

b 

1 

2 

9 

Do  Chores 

u 

4 

Help  a  Frienu  With  a  Problem 

i 

3 

Be  Sociable 

1  1 

9 

11 

Be  Nice  to  Family  Members 

n 

13 

Be  Tolerant  with  Family  Members 

4 

; 

4 

Be  ;t  Good  Parent 

2 

2 

Be  Kind  and  Considerate  of  Others 

1 

9 

10 

Experience  Life  Fully 

1  2 

1 

4 

Improve  oneself  (Qualities) 

10 

10 

Study  Hard  to  Get  Good  Grades 

II 

10 

1 

11 

Graduate  In  Allotted  Time 

4 

4 

Learn  for  Learning's  Sake 

1 

2 

1  2 

6 

Learn  New  Skills 

3 

3 

Get  Married 

1 

1 

Participate  In  Organizations 

6 

6 

(extracurricular  activities) 

Miscellaneous 

1 

1 

Go  to  Church 

2 

2 

Be  Attractive 

5 

5 

Attend  Cultural  Events 

1 

i. 

3 

Break  Bad  Habits 

3 

3 

TOTALS 

b  4  4  20  12  4 

20 

17  4  5  5  9  2  3 

115 
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Each  classification  should  be  chocked  b>  having  another  team 
sort  the  cards  *nto  the  categories  generated  by  the  original 
team. 

The  classifiers  (including  the  team  mentioned  in  (4)  above) 
should  rank  the  category's  for  importance  (or  wnatever  variable 
is  being  investigated) .  Note  that  this  is  a  check  cn  ranking 
by  frequency. 
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VIT.  VALIDITY  CHECK 

As  an  estimate  of  validity  ("Does  the  instrument  do  what  it  purports 
to  do?"),  test  sessions  »ere  run  to  see  if  ECHO-generated  data  represented 
important  and  prevalent  values  in  the  subject  culture.  The  sessions  also 
allowed  the  assessment  of  the  method's  sensitivity:  could  it  distinguish 
among  similar  subcultures?  In  addition,  the  test  sessions  were  used  to 
determine  how  to  select  specific  value  statements  from  the  many  categories 
supplied  by  the  classifiers.  Six  groups  were  tested:  UCLA,  Carnation, 
Prudential,  Northwestern,  Nurses,  and  Cuban  Exiles. 

A.  PROCEDURE 

Although  each  session  differed  in  some  way  from  the  others,  the 
same  basic  procedure,  a  "message  f'-sion",  was  followed  in  all.  The 
subjects  were  given  test  booklets  that  contained,  in  paired-comparison 
format,  equal-length  "messages"  (lists  of  category  titles)  that  came 
from  three  sources:  (1)  classification  sessions  that  represented  the 
subject  population;  (2)  classification  sessions  of  other  populations;  and 
(3)  a  list,  in  the  same  Jorm,  of  items  prepared  by  an  "expert"  in  that 
culture.  f>s  were  .required  to  select  the  one  of  each  pair  that  they 
believed  to  be  the  "most  important".  Figure  2  is  a  sample  page  from  a 
message-session  test  booklet. 

The  position  of  each  pair  of  messages  on  a  page  was  alternated  and 
the  pages  were  placed  in  random  order  in  the  test  booklets  to  control 
position  and  ordering  effects.  Eacu  message  from  a  classification  session 
c< nsisted  of  ti  titles  of  the  five  tegories  with  the  highest  frequency, 
in  rank  order. 

In  some  cases,  special  test  booklets  were  prepared  for  different 
subgroups.  For  example,  the  Carnation  subjects  were  divided  into  two 
groups,  secretaries  and  non-secretaries,  and  different  messages,  based 

The  term  "message"  was  inherited  from  an  early  study;  it  is  now  used  to 
refer  to  the  list  of  words  or  phrases  used  in  a  validity  check  session. 


UNCLASSIFIED 


17 


UNCLASSIFIED 


£1 

TO  GET  BETTER  GRADES 
TO  HELP  OTHER  PEOPLE 

TO  BE  KIND,  CONSIDERATE  AND  UNDERSTANDING  OF  OTHERS 
TO  BECOKu  A  SELF-FULFILLED  PERSON 
TO  UNDERSTAND  MYSELF  BETTER 


R61 

HAVE  FUN  AND  GET  ENJOYMENT  OUT  OF  LIFE 
STUDY  AND  GET  GOOD  GRADES 
MAKE  LOVE  AND  HAVE  SEXUAL  RELATIONS 
MAINTAIN  GOOD  HEALTH 

FEEL  AND  ACCEPT  SOCIAL  RESPONSIBILITY,  BE  A  REFORMER 


WRITE  YOUR  ANSWER  HERE _ .  Go  on  to  the  next  page. 


Figure  2.  Sample  Page  From  a  Northwestern  Test  Session  Booklet.  FI  is 
From  UCLA  Female  Data;  R61  is  From  Northwestern  Male  Data 
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on  secretarial  and  non-secretarial  data  respectively,  were  prepared  for 
them.  Similarly,  the  male  subjects  in  the  Cuban  and  Northwestern  groups 
received  test  booklets  containing  messages  based  mainly  on  the  male 
responses  while  the  females  in  those  two  groups  received  messages  created 
mainly  from  female  responses. 

The  hypothesis,  as  stated  earlier,  was  that  Ss  would  select  the 
list  which  came  from  their  own  population  over  any  other,  and  would  select 
lists  which  came  from  a  similar  population  over  lists  coming  from  "alier." 
populations.  "Population",  in  this  instance,  refers  to  both  the  data 
source  and  the  classifiers.  Table  4,  in  the  example  below,  shows  some 
of  the  possible  variations  of  cla.,jifier  and  data-source  mixes. 

B.  MESSAGE  CONSTRUCTION 

Several  decisions,  based  on  the  judgment  of  the  investigators,  were 
made  about  message  construction:  (1)  the  language  of  the  subjects  should 
be  used  unchanged;  (2)  the  selection  of  the  specific  items  to  be  included 
should  be  by  mechanical  means  and  independent  of  the  ju  igment  of  the 
investigators;  (3)  messages  should  contain  approximately  five  items.  In 
accordance  with  the  first  decision,  no  connecting  words  or  phases  could 
be  used,  so  category  titles  were  presented  in  a  list.  To  implement  the 
second  decision  several  methods  were  tried;  at  the  present  time  the 
evidence  seems  clear  that  the  categories  with  the  highest  frequencies 
(the  largest  number  of  cards)  are  the  most  representative.  Apr  ndix  II 
contains  data  supporting  that  contention.  The  other  methods  tried  were 
(1)  ranking  by  classifiers  (high  correlation  with  frequency),  (2)  ranking 
by  j>'s  estimate  of  importance,  and  (3)  selection  by  source;  it  is  possible 
that  this  last  method  may  have  merit  for  other  research  purposes. 

The  third  decision,  to  limit  the  number  of  categories,  fit  the 
empirical  findings:  in  almost  every  classification  session,  the  five 
categories  with  the  highest  frequencies  accounted  for  over  60%  of  the 
cards.  In  the  few  exceptions,  the  sixth  category  was  tied  with  the  fifth. 
When  a  tie  occurred,  a  coin  was  tossed  to  determine  the  rank. 
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An  "expert"  was  asked  to  write,  in  rank  order  of  importance,  the 
significant  components  of  the  clerical  jobs  in  his  company.  An  expert 
was  defined  as  being  a  person  with  considerable  knowledge  about  a 
population  but  who  was  not  part  of  that  population.  An  example  was  the 
personnel  manager  of  Prudential  relative  to  the  clerical  force  of  that 
company . 

C.  EXAMPLE  OF  RESULTS 

The  details  of  each  validity  cheer  session  will  be  found  in  Appendix 
I;  the  Cuban  study  is  reported  here  as  an  example. 

1,  List  Construction 

The  Cuban  lists  consisted  of  the  titles  of  the  five  highest-frequency 
categories  in  each  of  six  classification  sessions.  Table  4  shows  the 
classification  systems  used. 


TABLE  4 

CUBAN  CLASSIFICATION  SYSTEMS  EMPLOYED  IN  CONSTRUCTION  OF  LISTS  FOR  VALIDITY 

TEST 

LIST  CLASSIFICATION  SYSTEM 

I  Males  classifying  male  positive  data 

III  Females  classifying  female  positive  data 

V  Males  classifying  combined  male  positive  and  negative  data 

VI  Females  classifying  combined  female  positive  and  negative 
data 

VII  Males  classifying  female  positive  data 

IX  Females  classifying  male  positive  data 


In  addition,  a  list  (tra  islated  into  Spanish)  was  constructed  from 
the  five  hignest-f requency  categories  in  the  Stanford  positive  data. 


20 


UNCLASSIFIED 


UNCLASSIFIED 


2.  Procedure 

The  lists  were  presented  in  a  paired-comparison  format,  the  Ss 
being  instructed  to  select  from  each  pair  the  list  which  they  considered 
to  be  "most  important". 

The  test  booklets  consisted  of  randomly  ordered  pages,  each  of 
which  contained  two  lists  and  an  answer  blank.  Figure  3  is  a  sample  page 
Male  and  female  jSs  received  two  slightly  different  test  booklets. 

3.  Results 

Table  5  indicates  the  proportions  of  JSs  selecting  each  of  the  lists 
and  the  levels  of  significance  of  the  results. 


XI_ 

AYUDAR  A  OTROS. 

INCREMENTAR  EL  BIENESTAR  COMUN. 
ESTAR  INTERESADO  POR  MUCHAS  COSAS. 
VIVIR  LA  VIDA  PLENITUD. 

ACTIVIDAD  FAMILIAR. 


Ill 

AYUDAR  AL  PROJIMO. 

LIBERACION  DE  CUBA. 

PROPAGACION  DE  LA  FE  CRISTIANA. 

RESPETO  A  LAS  LEYES  Y  BUEN  COMPORT AM I ENTO  EN  ESTE  PAIS. 
COMBAT  I R  EL  COMUNISMO. 


RESPUESTA 


Figure  3.  Sample  Page  From  Cuban  Test  Booklet.  XI  is  a  Translation  of 

Stanford  Data;  III  is  Cuban  Female  Data 
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J3s  appeared  to  discriminate  between  the  lists  on  the  basis  of  the 
sex  and  the  culture  of  the  subjects  from  whose  responses  the  lists  were 
generated.  For  example,  though  the  male  Ss  rejected  their  own  list  (I) 
in  favor  of  that  generated  by  the  Cuban  females  (III),  they  significantly 
chose  their  own  lists  over  that  based  on  the  Stanford  data  (XI) .  These 
results  also  present  some  evidence  on  the  question  of  whether  positive 
and  negative  responses  should  be  classified  together  or  separately.  As 
can  be  seen  in  Table  5,  both  male  and  female  subjects  selected  their  own 
positive  lists  over  those  generated  by  the  classification  of  combined 
positive  and  negative  responses. 

The  validity  test  results  indicate  tht  the  ECHO  instrument  can 
determine  not  only  important  attitudes  within  the  subject  population  but 
also  m^ny  of  the  differences  among  various  subgroups  within  the  population. 
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VIII.  POWER  STRUCTURE  ANALYSIS 

The  basic  question,  "What  is  a  good/fcad  thing  to  do?"  has  a  second 
part:  "Who  would  approve?"  The  answers  to  this  question,  which  we  call 
"sources"  and  sociologists  call  "surrogates,"  provide  additional  infor¬ 
mation  about  the  perceived  "reinforcers"  which  control  behav.or  in  the 
subject  culture.  A  tenable  assumption  is  that  human  beings,  lik?.  ail 
animal  life  from  one-celled  amoebae  to  Nobel-prize-winning  physicists, 
tend  to  seek  positive  reinforcement  (reward)  and  avoid  negative  reinforce¬ 
ment  (punishment);  those  who  dispense  these  reinforcements  hold  "power" 
in  the  culture.  People  acquire  the  beliefs,  attitudes,  norms,  and  values 
of  their  reference  culture  through  a  process  of  learning,  i.e.,  by 
experiencing  the  positive  and  negative  consequences  of  their  behaviors. 

This  learning  process  is  refen.-  :o  as  "socialization." 

The  source (s)  which  control  a  behavior  may  change  over  time.  In 
most  cases,  the  source  of  approval  or  disapproval  changes  from  an  external 

* 

entity  to  an  internal  control  mechanism,  i.e.,  people  learn  to  feel  guilty 
when  they  do  something  that  their  society  says  is  wrong.  The  Power  Struc¬ 
ture  Analysis  element  of  the  ECHO  methodology  permits  identification  of  the 
perceived  sources  of  reinforcement  and  changes  in  the  structure,  or  pattern, 
of  power  impinging  on  the  individual.  Our  studies  indicate,  for  instance, 
that  children  in  the  third  grade  tend  to  view  their  parents,  particularly 
mother,  as  the  most  important  source  of  approval  and  disapproval.  College 
students,  on  the  hand,  cite  "self"  as  the  most  important  source.  The 
college  students  appear  to  have  "internalized"  the  value  system  of  the 
culture;  the  third  graders  still  use  their  parents  to  tell  them  what 
society  thinks  is  right  or  wrong.  Sixth  grade  children  respond  more  like 
the  college  students.  Figure  4  shows  the  relative  degree  of  inPuence 


Note  that  "guilt"  is  what  a  person  experiences  when  he  knows  hat  he  has 
done  something  wrong;  "shame"  is  what  he  feels  if  anyone  else  knows 
about  it.  The  two  feelings  are  not  necessarily  the  same. 
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Figure  A.  Relative  Influence  of  "Self"  and  "Parents"  At  Three  Grade  Levels 
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that  "seif"  and  "parents"  exercise  over  the  behaviors  of  the  individuals 
in  our  test  populations;  the  increasing  frequency  of  the  term  "self"  may 
well  reflect  the  process  of  "internalization"  (or  'n  sociological  terms 
"interiorization") . 

Implications  for  behavior  modification  immediately  suggest  themselves. 
For  instance,  data  collected  so  far  indicates  that  the  parents,  and  partic¬ 
ularly  the  mother,  are  overwhelmingly  the  significant  figures  in  the  life 
of  a  child.  It  follows  that  if  one  wishes  to  be  effective  in  modifying 
the  behavior  of  a  disturbed  child,  he  must  deal  with  the  parents,  either 
by  minimizing  their  impact  or  modifying  their  behavior.  This,  indeed,  is 
what  child  therapists  do:  more  time  is  spent  with  the  parents  of  a 
disturbed  child  than  with  the  child  himself.  Or,  looking  at  the  Cuban 
data,  one  might  hypothesize  that  if  the  behaviors  of  Cuban  women  are  to 
be  modified,  the  Church  would  be  a  potent  force;  whereas  for  Cuban  men  it 
would  be  less  so.  Evidence  for  this  statement  comes  from  the  frequent 
mention  by  Cuban  females  of  church  and  church-related  reinforcers. 

The  Power  Structure  Analysis  is  also  capable  of  detecting  cultural 
differences  in  sources  of  reinforcement.  For  instance,  "Anglo"  university 
students,  both  male  and  female,  cite  "myself"  as  source  of  approval  far 
more  often  than  Cuban  subjects  do  (see  Table  6).  Note  too  that  males,  in 
both  cultures,  cite  "myself"  more  often  than  do  their  female  counterparts; 
the  degree  of  difference  between  males  and  females  on  this  dimension  might 
be  thought  of  as  an  index  of  the  cultures'  belief  in  the  "equality"  of  the 
masculine  and  feminine  roles. 


The  comments  on  "value  internalization"  and  the  "socialization  process" 
are  based  on  the  findings  of  our  research  with  two  populations  o'  ele¬ 
mentary  school  children.  These  experiments  were  designed  to  answer  two 
major  questions:  (1)  How  old  must  our  subjects  be  to  use  the  standard 
IBM  card  form  of  the  instrument?  and  (2)  Can  we  detect  the  changes  which 
accompany  (or  constitute)  the  socialization  process?  A  complete  report 
of  this  phase  of  the  research  is  contained  in  Appendix  III. 

The  literature  is  replete  with  evidence  that  supports  this  hypothesis; 
e.g.,  see  Area  Handbook  for  Cuba,  Special  Operations  Research  Office, 
America University,  1961,  p.  138. 
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TABLE  6 

F\EQUENCY  OF  OCCURRENCE  OF  "MYSELF"  AS  SOURCE  OF  APPROVAL  OR  DISAPPROVAL 

Ref. 


Code 

Subject  Population 

Total  N 

f  of  "Myself 

%  of  N  "Myself 

Al 

Stanford  Positives 

700 

258 

36.9 

A2 

Stanford  Negatives 

695 

226 

32.5 

P.+ 

UCLA  Positives  (all  Ss) 

1169 

387 

33.1 

B- 

UCLA  Negatives  (all  Ss) 

NOT  AVAILABLE 

B+ 

UCLA  Female  Positives 

506 

151 

29.8 

B+ 

UCLA  Male  Positives 

663 

235 

35.4 

CE1  -  CUBAN  DATA 

I  &  III  Cuban  Positives  (all  Ss) 

259 

36 

13.9 

II  &  IV  Cuban  Negatives  (all  Ss) 

241 

41 

17.0 

I 

Cuban  Male  Positives 

150 

.  31 

20.7 

II 

Cuban  Male  Negatives 

140 

24 

17.1 

III 

Cuban  Female  Positives 

109 

5 

4.6 

IV 

Cuban  Female  Negatives 

101 

17 

16.8 

A  behavior  may  be  reinforced  by  two  completely  different  sources 
in  the  same  culture,  and  the  ECHO  power  structure  analysis  can  detect 
this  fact.  For  example,  the  UCLA  students  saw  "getting  married"  as  a 
good  thing  to  do.  When  asked,  "Who  would  approve?",  the  men  said  "myself" 
and  the  women  said  "my  parents"  (see  Table  7). 

TABLE  7 

UCLA  STUDENTS:  "IT'S  A  GOOD  THING  TO  GET  MARRIED" 

Gender  cf  Ss 


SOURCE  OF  APPROVAL 

MALES  1 

1  FEMALES 

FAMILY 

1 

10 

11 

MYSELF 

9 

1 

10 

10 

11 

21 

X  *  13.61  with  v  *  1  is  significant  at  a  *  .u01+ 
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The  Power  Structure  Analysis  element  of  FCHO  technique  appears  to 
be  a  powerful  tool  for  describing  and  understanding  populations.  The 
usefulness  of  this  review  will  be  greatly  increased  in  the  future  when 
the  relationship  between  specific  values  end  sources  is  better  understood. 
Subsumed  in  this  relationship  is  an  understanding  of  which  values  go 
together  in  natural  clusters  c  constellations.  Exerting  pressure  for 
change  on  one  element  of  a  cluster  will  cause  a  reaction  in  the  other 
elements  of  the  cluster;  understanding  the  relationship  between  values 
will  permit  greater  accuracy  in  predicting  change  (and  therefore  greater 
accuracy  in  influencing  change) .  Source  lvalue  relationships  are 
endogenous  to  this  value  cluster  problem. 
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IX.  DATA  PROCESSING 

This  section  is  divided  into  three  subsections,  each  of  which  treats 
a  different  facet  of  the  data  processing  procedures  used  in  Phase  I.  The 
first  subsection  explicates  the  "Operational  Package"  which  is  used  to 
transform  the  raw  inputs  into  value  hierarchies  and  "power  structures"; 
the  second  subsection  lists  the  library  programs  which  have  been  developed 
for  general  use  under  the  terms  of  the  ECHO  contract.  Possible  applications 
of  recent  advances  in  computer  technology  are  considered  in  the  last 
subsection. 

Automatic  data  processing  by  high-speed  computers  is  an  important 
element  of  the  ECHO  methodology  which  allows  reliable,  unbiased  data 
analysis  while  minimizing  time  and  cost.  All  of  the  computer  operations 
in  the  project  were  completed  on  a  CDC  3600  computer,  operated  by  the 
Computer  Center  of  Santa  Barbara,  a  General  Research  Corporation  subsidiary. 
The  computer  programs  used  in  Phase  I  of  Project  ECHO  are  of  two  types: 
Operational  Package  Programs  and  Library  Programs. 

A.  "OPERATIONAL  PACKAGE"  PROGRAMS 

Each  of  these  39  programs  performs  the  standard  analytical  computa¬ 
tions  that  all  raw  ECHO  inputs  undergo  as  part  of  the  "Operational  Package" 
processing. 

KOUNTEM — computes  the  frequency  of  all  category: source  combinations. 
The  program  was  modified  to  allow  comparison  of  different  classifications 
of  the  same  data  cards.  Each  of  the  eight  versions  provides  information 
about 

1.  The  number  of  cards  (N)  in  the  data  pool 

2.  The  number  of  responses  '.n  each  category,  i.e.,  the  prevalence 
of  the  value  in  the  hierarchy  of  the  target  populations 

3.  The  relative  importance  of  each  source  associated  with  a  given 
value 
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TRANSVES — compares  any  two  classification  systems  which  are  based, 
in  whole  or  in  part,  on  the  same  set  of  responses.  The  output  from  this 
program  answers  the  question,  "What  did  two  different  groups  of  Ss  say 
about  the  same  cards?"  This  prograir  has  28  versions  now  in  operation. 

SUBJSOUR — identities  the  frequency  of  sources  cited  by  each  subject 
Information  from  this  output  can  be  used  in  answiring  questions  about  the 
power  structure  impinging  on  any  given  subject  in  the  population. 

POWERSTRC — quantifies  the  relative  "power"  each  source  has  over  a 
given  behavior  in  the  population's  "collective  repertoire. " 

Example:  Male  and  female  college  students  expect  different  people 
to  approve  if  they  "get  married,"  as  the  following  POWERSTRC  output 
demonstrates.  (Table  7,  above,  is  an  abridgement  of  this  output.) 


Source  of 
Approval 

Ss  Gender 

Male  j  Female 

Myself 

9 

1 

Parents 

1 

10 

Mate 

2 

- 

Society 

1 

- 

Totals 

13 

11 

WRECKEM — Computes  the  frequency  of  every  combination  of  descriptions 
assigned  during  several  classifications  of  the  same  data.  This  process 
facilitates  the  comparison  of  several  groups'  opinions  of  the  same  cards 
in  cases  where  more  than  two  teams  classified  the  same  data.  Representing 
the  data  in  matrix  form  when  there  are  more  than  two  nominally  scaled 
dimensions  is  not  feasible. 
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B.  LIBRARY  PROGRAMS 

Several  other  programs  were  written  especiaJly  for  the  CDC  3600 
computer  used  in  the  data  analysis  poition  of  the  project.  This  library 
includes  the  following  programs: 

1 .  Central  Tendency  Statistics 

a.  mean  (x) 

b.  mode 

c.  median 

2 .  Dispersion  (Variability)  Statistics 

a.  sample  variance 

b.  sample  standard  deviation 

c.  third  moment  about  the  mean 

d.  single  factor  analysis  of  variance 

3.  Correlation  Statistics 

a.  Pearson  Coefficient  of  Correlation  (p) 

b.  Spearman's  Rho  (p)  for  Ranked  Data 

c.  Kendall  W  for  Ordinal  Data 

4 .  Chi-Square  Tests 

a.  r*K  multifactor  Chi-Square  Test  with  (r  -  1) (K  -  1) 
degrees  of  freedom 

b.  Single-sample  Chi-Square  Test  with  (K  -  1)  degrees  of 
freedom 

5 .  "Student's  't'  Test"  for  Sample  and  Population  Data 

C.  OTHER  POSSIBLE  APPLICATIONS  OF  COMPUTER  TECHNOLOGY  TO  THE  ECHO 
METHODOLOGY 

Three  computer  innovations  have  been  considered  as  being  of  possible 
value  to  the  ECHO  methodology:  (1)  Computerizing  the  classification  process 
(2)  generating  the  exact  vocabulary  of  our  subjects  in  concordance  form; 
and  (3)  identifying  the  common  syntactical  style(s)  of  the  population. 

The  latter  two  ideas  hold  promise  as  possible  beginnings  of  a  computerized 
persuasive  message  generation  system. 


UNCLASSIFIED 


31 


UNCLASSIFIED 


The  possibility  of  computerizing  the  classification  system  has 
been  investigated  and  rejected.  Indigenous  classifiers  appear  to  be  an 
important  element  of  the  ECHO  methodology;  even  without  this  technical 
constraint,  the  cost  of  such  an  operation  would  be  prohibitive. 
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X.  SUMMARY  OF  MAJOR  PHASE  I  FINDINGS 

A  brief  synopsis  of  some  Phase  I  results  follows. 

A.  SAMPLE  SIZE  REQUIREMENTS 

Outputs  from  15-20  people  have  been  used  to  accurately  reflect  the 
value  structure  of  a  100  to  200  person  group.  Insofar  as  this  larger 
group  is  representative  of  a  still  larger  group  (a  university,  ten-year 
old  children,  a  country),  the  very  small  sample  can  yield  information,  in 
a  brief  period  of  time,  formerly  available  only  from  experts  who  had 
spent  years  studying  that  population.  LCHO  information  is  current;  the 
"expert"  may  be  obsolete  without  knowing  it.  The  accuracy  of  the  general¬ 
ized  results  varies  as  a  function  of  target  group  homogeneity,  represent¬ 
ativeness  of  the  sample,  motivation  and  fatigue  of  the  indigenous  classi¬ 
fiers,  and  the  prevalence  of  the  value  under  study.  The  more  prevalent 
the  value  in  the  target  group's  value  hierarchy,  the  fewer  subjects  are 
required  to  detect  this  fact.  Conversely,  a  large  number  of  subjects 
are  needed  to  pinpoint  values  held  by  small  segnr  :s  of  the  population 
under  study. 

B.  MEASURES  OF  PREVALENCE  AND  INTENSITY 

Evidence  to  date  indicates  that  frequency  of  mention  (how  often 
a  given  value  is  cited)  reflects  the  prevalence  of  the  value  in  the 
target  population's  macro-value  structure.  In  addition,  frequency  may 
imply  intensity,  but  this  conclusion  must  be  reached  with  caution, 
because  high  frequency  can  also  be  caused  by  saliency  (i.e.,  the  par¬ 
ticular  value  stands  out  clearly  to  the  subjects)  and  "comfort"  (i.e., 
the  value  is  well-accepted  and  the  subjects  feel  comfortable  in  talking 
about  it). 

C.  TIME  FACTORS  IN  DATA  COLLECTION  PROCEDURES 

U.S.  adult  Ss  require  20  to  35  minutes  to  answer  20  ECHO  questions. 

A  group  of  third  grade  pupils  took  approximately  one  and  a  half  hours  to 
do  the  same  thing.  (An  equivalent  group  of  sixth  graders,  however,  com¬ 
pleted  the  task  in  the  same  time  as  adults.) 
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D.  HOW  MANY  VALUES  CAN  A  SINGLE  EXPERIMENT  DETECT? 

The  number  of  categories  in  a  classification  system  is  a  function 
of  the  number  of  cards,  i.e.,  the  framework  required  to  organize  the 
cards  varies  directly  with  the  number  of  cards  classified.  It  also  varies 
with  the  "set"  of  the  classifiers,  which  may  be  influenced  by  natural 
tendencies,  assumptions,  or  instructions. 

E.  QUANTIFYING  DIFFERENCES  IN  SOCIAL  PERCEPTION 

Differences  In  how  two  populations  view  the  same  behavior  can  be 
detected  and  quantified  by  having  a  single  batch  of  data  classified  by 
Gs  from  both  groups. 

F.  REDEFINING  "VALUE  DEFINITIONS" 

Meanings  assigned  by  a  group  of  indigenous  Cs  can  be  redefined 
by  having  several  other  groups  of  _Cs,  from  the  same  population,  classify 
the  data. 

G.  IMPORTANCE  OF  A  PARTICULAR  VALUE  IN  THE  S’s  VALUE  HIERARCHY 
The  opinions  of  individual  Ss  about  the  importance  of  a  value 

may  show  wide  variability.  Ss'  rankings  of  item  "importance"  (I)  are 
directly  related  to  the  sequence  of  response;  (I)  values  are  statisti¬ 
cally  independent  of  categories,  i.e.,  importance  rankings  are  evenly 
distributed  among  all  categories. 

H.  DIFFERENTIATING  BETWEEN  POPULATIONS  AND  SUB-POPULATIONS 

Value  hierarchies  discovered  by  ECHO  differentiate  between  different 
populations  (e.g.,  Cubans  and  U.S.  college  students)  and  between  sub¬ 
populations  in  a  single  culture  (e.g.,  males  and  females). 

I.  VALUE  OF  INDIGENOUS  CLASSIFIERS 

Experiments  in  which  teams  from  different  populations  classified 
data  from  a  single  population  demonstrated  the  value  of  indigenous 
classifiers;  these  individuals  are  able  to  impart  subtle  meanings  and 
differentiation  to  the  data  that  "experts"  often  overlook. 
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J.  CLASSIFICATION  PROCEDURES 

The  current  operational  procedure  for  categorizing  the  data 
requires  that  (1)  each  data  pool  be  classified  by  two  different  teams 
of  £s;  (2)  the  categories  be  ranked  on  an  "importance  scale"  by  the 

Cs\  and  (3)  the  cards  be  sorted  into  a  predetermined  set  of  categories 
as  a  test  of  classifier  reliability. 

The  data  are  divided  into  positive  and  negative  (i.e.,  good  and 
bad  thing  to  do)  samples,  and  classified  as  separate  samples  rather  than 
as  a  single  "mixed"  sample.  Categories  from  a  single-sample  classifi¬ 
cation  system  are  preferred  in  validity  tests  over  those  from  "mixed" 
classification  systems. 

Data  from  20  to  30  subjects  (200-300  cards)  can  be  classified  by 
one  team  at  one  sitting  before  fatigue  and  boredom  introduce  added 
variance  to  the  process.  Classifier  reliability,  i.e.,  the  agreement 
between  two  teams  of  indigenous  classifiers,  is  high  under  the  conditions 
described  above. 

K.  INDICATORS  OF  CLASSIFIER  EFFICIENT 

Several  potential  indicators  of  classifier  efficiency  are  under 
consideration.  Identifying  and  training  good  indigenous  Cs  should 
greatly  improve  the  efficiency  of  the  data  analysis  procedure. 

L.  ABILITY  OF  EC’.iO  TO  OPERATE  IN  NON-ENGLISH-SPEAKING  CULTURES 

The  ECHO  methodology,  as  we  know  it  now,  is  capable  of  operating 
effectively  in  literate  non-English-speaking,  Occidental  populations 
(e.g.,  Spanish /Cuban) .  The  ability  of  the  method  to  generalize  beyond 
Judeo-Christian  cultures  to  other  cultures  (e.g.,  Thai)  will  be  evalua¬ 
ted  in  Phase  II.  The  structure  of  Oriental  languages  and  the  impact  of 
a  different  religious  philosophy  may  require  a  modification  of  the  ECHO 
methodology. 
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M.  OPERATIONAL  PROCEDURES  FOR  ILLITERATE  SUBJECTS 

ECHO  inputs  can  be  collected  from  nonhostile  illiterate  subjects 
by  oral  interviews.  However,  the  data  collection  interview  can  be  an 
extremely  time-consuming  procedure.  For  example,  subjects  have  taken  an 
hour  and  20  minutes  to  give  10  "good"  and  10  "bad"  responses.  This  seems 
to  be  a  function,  not  of  difficulty,  but  of  the  subject's  desire  to  talk 
to  someone  whc  will  listen. 

Several  methods  of  gathering  and  classifying  oral  interview  data 
are  under  consideration. 

N.  WHAT  IS  THE  MINIMUM  AGE  OF  SUBJECTS  WHO  CAN  USE  THE  WRITTEN 
INSTRUMENTS? 

School  children  as  young  as  8  years  old  are  capable  of  handling 
the  IBM  card  form  of  the  ECHO  instrument;  however,  they  require  1  to 
1-1/2  hours  to  complete  20  cards. 

O.  SENSITIVITY  TO  DYNAMIC  CHANGES 

The  ECHO  methodology  appears  to  be  capable  of  detecting  dynamic 
changes  in  the  macro-value  structure  of  populations.  This  will  provide 
a  sensitive  instrument  for  tracing  the  internalization  of  particular 
values  and  the  socialization  process  in  a  culture. 

P.  "POWER  STRUCTURE"  ANALYSIS 

Perceived  sources  of  positive  and  negative  reinforcement  operating 
on  a  population  (the  "power  structure")  can  be  identified  and  quantita¬ 
tively  described  by  the  current  ECHO  analysis  system. 
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XI .  implications  and  further  studies 

The  project  to  date  has  been  designed  to  perfect  the  methodology 
and  to  determine  its  feasibility.  The  results  have  seemfd  impressive, 
but  the  number  of  subjects  on  some  of  the  critical  tests  has  been 
small.  Should  ECHO  be  used  in  the  field,  the  stakes  may  be  very  high; 
prudence  dictates  that  new  tests  be  made  and  the  old  ones  replicated. 

The  following  topics  are  among  those  which  must,  or  might,  be  given 
consideration  in  future  research  efforts. 

A.  MODIFYING  THE  METHODOLOGY  FOR  USE  WITH  ILLITERATE  POPULATIONS 

All  subjects  to  date  (with  the  exception  of  those  participating 
in  the  East  Los  Angeles  pilot  study)  were  literate  in  English  or 
Spanish;  applying  the  ECHO  technique  to  illiterate  populations  will 
require  some  major  modifications  in  the  methodology.  These  modifica¬ 
tions  seem  warranted  in  view  of  the  large  proportion  of  the  potential 
subject  populations  which  is  functionally  illiterate.  Several  questions 
must  be  answered:  How  can  data  be  collected  from  illiterates?  What 
system  of  data  classification  can  be  developed  to  make  use  of  indigenous 
classifiers  and  the  subtle  meanings  they  can  impart  to  the  final  classi¬ 
fication  system?  How  can  tentative  findings  be  fed  back  to  the  popula¬ 
tion  for  validation?  Several  potential  solutions  to  these  problems 
immediately  suggest  themselves.  Data,  for  instance,  could  be  collected 
in  oral  interviews,  but  what  influence  will  loss  of  subject  anonymity 
have  on  the  type  of  values  the  Ss  mention? 

For  every  potential  modification  of  the  current  methodology  there 
is  an  effect;  experiments  must  be  undertaken  to  discover  the  optimal 
way  (given  certain  economic,  efficiency,  and  social  criteria)  of  collect¬ 
ing  and  classifying  meaningful  data  from  functionally  illiterate  sub¬ 
jects. 


B.  UTILIZING  "SECONDARY"  DATA 

The  current  ECHO  methodology  makes  use  of  approximately  60  to  70 
percent  of  all  the  data  collected,  i.e.,  about  60  to  70  percent  of  the 
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data  falls  in  the  five  most  frequently  mentioned  categories  of  each 
system.  The  remaining  data  probably  hold  much  significant  information 
particularly  about  small  subcultures,  fringe  groups,  and  socially 
unpopular  or  forbidden  behaviors.  In  addition,  the  individual  responses 
within  the  five  major  categories  may  yield  important  information. 

C.  VARIATIONS  OF  THE  PROJECTIVE  SURVEY  QUESTIONNAIRE 

The  ECHO  technique  is  not  limited  to  a  single  form  of  the  pro¬ 
jective  survey  question;  the  principle  underlying  the  technique  can 
be  used  in  many  different  ways  to  discover  many  different  types  of  data. 

The  majority  of  our  experience  has  been  with  the  question:  "What 
is  a  good/bad  thing  to  do  and  who  would  approve/disapprove?"  Some 
preliminary  work  has  been  done  with  the  question:  "What  is  a  good/bad 
thing  that  could  happen  and  what  (or  who)  would  be  chiefly  responsible 
for  its  happening?"  The  first  question  seeks  to  identify  values  held  by 
the  target  population,  while  the  latter  attempts  to  tap  the  population's 
expectations  and  perceptions  of  causation  (internal  or  external  causes). 
Several  variations  of  the  ECHO  question  have  been  formulated,  and 
research  will  be  needed  to  evaluate  their  effectiveness  in  achieving 
their  intended  aims  (e.g.,  detection  of  areas  of  conflict  between  an 
individual  and  his  society  or  sources  of  conflict  between  small  sub¬ 
cultures  and  the  larger  society). 

D.  USE  OF  CHILDREN  AS  INDICATORS  OF  PARENTS'  CULTURE 

Phase  I  pilot  studies  indicate  that  children  eight  years  old  and 
older  are  capable  of  handling  the  standard  IBM  form  of  the  questionnaire; 
however,  little  is  known  about  their  ability  to  classify  the  data. 
Experiments  to  ascertain  this  information  should  be  initiated.  By  break¬ 
ing  the  classification  process  down  into  sub. asks,  and  using  children  as 
Cs,  one  should  be  able  to  identify  the  potential  sources  of  difficulty 
when  the  classifiers  are  functionally  illiterate  or  minimally  literate. 
Research  with  children  is  preferable,  in  some  cases,  to  experiments  with 
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adult  Ss:  children  are  reliable  (e.g.,  not  one  single  child  failed 
to  complete  all  20  cards;  a  number  of  Ss  in  every  other  population  did 
leave  several  cards  blank),  children  are  less  suspicious  and  more 
cooperative,  and  test  populations  of  children  are  usually  available  over 
a  long  period  of  time  (e.g.,  one  semester,  one  school  year).  ECHO  data 
collected  from  children  also  appear  to  be  useful  as  indices  of  the 
culture's  social  control  system  and  the  value  system  of  the  parents' 
society.  Research  should  be  undertaken  to  discover  (1)  the  extent  to 
which  data  can  be  collected  and  classified  by  children  and  (2)  the 
nature  of  the  linkage  between  children's  perceptions  of  their  parents' 
society  and  reality, 

E.  CREATION  OF  MESSAGES  FROM  ECHO  OUTPUTS 

The  value  hierarchies  produced  by  the  classifiers  are  independent 
declarative  sentences:  "It  is  a  good  thing  to  ...."  or  "It  is  a  bad 
thing  to  ....".  No  attempt  was  made  during  Phase  I  research  to  link 
different  values  together  to  form  credible  messages-  pilot  work  with 
ECHO  messages  during  the  summer  of  1956  indicated  clearly  that  credible 
messages  can  be  generated  from  the  classifiers'  outputs.  Which  values 
go  together  to  form  credible  persuasive  communications  must  be  discovered 
likewise,  the  relationships  between  the  source  (implicit  rc  explicit) 
of  the  communication  and  the  values  that  should  be  included  in  the 
message  must  be  investigated.  Successful  completion  of  this  research 
could  produce  a  technique  for  generating  messages  which  are  superior, 
in  credibility  or  persuasiveness,  to  those  generated  by  experts.  Couch¬ 
ing  the  communications  in  the  vocabulary  and  syntax  which  are  popular 
with  the  target  population  should  increase  the  power  of  the  messages. 

The  data  needed  to  complete  this  operation  are  produced  through:  (1) 
the  actual  data  collected  from  Ss  and  (2)  the  classifiers'  definitions 
of  the  categories.  Computerized  concordance  and  parsing  programs,  when 
applied  to  ECHO  data,  have  potential  as  objective,  unbiased  sources  of 
guidance  for  writers  of  persuasive  messages. 
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F.  BEHAVIOR  MODIFICATION  AND  ATTITUDE  CHANGE 

Messages  produced  ii  th_-  manner  described  above  might  be  used  to 
modify  the  behavior  of  cer  ain  populatior s.  Applying  the  ECHO  outputs 
to  behavio  and  attitude  change  tasks  will  require  a  more  sophisticated 
technique. 
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APPENDIX  I 

A  DETAILED  STATEMENT  OF  HISTORY 

The  ECHO  methodology  has  oeen  tested  and  refined  on  several  differ¬ 
ent  subject  populations.  The  purpose  of  this  appendix  is  to  trace  the 
development  of  ECHO,  describing  the  basic  methodology  employed  with  each 
test  population  and  the  impact  which  each  study  had  upon  subsequent 
applications  of  ECHO. 

Sections  A  and  B  describe  the  pilot  work  of  Summer,  1966.  The  rest 
of  the  appondix  describes  the  project  since  January  1967. 

A.  GENERAL  RESEARCH  CORPORATION  SECRETARIES 

1 .  Subjects  (Ss) 

Twenty  female  secretaries  served  as  subjects.  The  population  was 
divided  into  two  samples  of  10  Ss,  the  first  group  serving  in  the  data 
collection  phase  and  the  second  group  in  the  message  session. 

2.  Data  Collection  Procedures 

In  an  individual,  verbal  interview,  each  subject  was  instructed  to 
make  as  many  responses  as  possible  to  the  following  questions: 

1.  What  could  a  petoon  like  yourself,  a  secretary  in  an  organi¬ 
zation  like  General  Research,  do  that  would  be  a  good  thing 
to  do  and  someone  would  praise  you  for  it?  Who  would  be  the 
person  or  persons  that  would  praise  you? 

2.  What  could  a  person  like  yourself,  a  secretary  in  an  organi¬ 
zation  like  General  Research,  do  that  would  be  a  bad  thing  to 
do  and  someone  would  reprimand  you?  Who  would  be  the 
person  or  persons  that  would  reprimand  you? 

These  resp'~  ses  were  recorded  in  writing  by  the  interviewers. 
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j.  Classification  Procedure 

The  10  generated  52  responses  to  the  positive  question  (1)  and 
72  responses  to  the  negative  question  (2).  The  interviewers  classified 
the  responses  by  creating  categories  which  represented  responses  of  simi¬ 
lar  content. 


Messages 


a.  Message  Construction 

Only  those  categories  mentioned  by  at  least  half  of  the  subjects 
were  considered  in  preparing  the  two  ECHO  messages.  These  were  the  follow¬ 
ing  categories : 

No.  of  Ss  No.  of  Mentions 

Positives : 

Personal  behavior  6  16 

Attitude  toward  work  7  13 

Attitude  toward  boss  5  11 


Negatives : 


Attitude  toward  work 
Personal  behavior 
Attitude  toward  boss 
Security 


10 

6 

7 

5 


21 

16 

11 

6 


The  interviewers  created  one  of  the  ECHO  messages  (ECHO  +)  by  linking 
together  statements  taker  directly  *rom  the  selected  categories.  Literary 
style  modifications  were  kept  to  a  minimum.  The  second  ECHO  message 
(ECHO  -)  was  composed  by  changing  positive  statements  into  negative  ones 
in  such  a  way  that  the  generated  message  would  hopefully  still  sound 
plausible  to  persons  outside  of  the  subject  nopulation.  An  i.tample: 

'*Take  the  initiative;  guess  ahead  what  he  will  need  instead  of  waiting  to 
be  told"  was  changed  to:  "It  is  not  appreciated  as  much  as  many  girls 
think  for  a  secretary  to  look  for  work  on  her  own  initiative.  Don't  try 
to  anticipate  problems  and  begin  work  before  it  has  been  asked  for." 
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Two  "expert"  mer sages  were  also  written,  one  by  the  head  secretary 
at  General  Research  and  the  other  by  a  secretary  at  the  University  of 
California  Li  Santa  Barbara. 

b.  Message  Session 

The  subjects  in  the  second  group  were  shown  the  four  messages, 
unlabeled  as  to  origin,  in  individual,  verbal  interviews  and  asked  to 
answer  the  following  questions: 

1.  These  four  messages  were  written  by  four  different  pern  •ms. 

Tell  us,  judging  from  what  they  wrote,  what  type  of  person 
do  you  think  each  of  them  is? 

2.  Only  one  of  these  messages  was  written  by  a  person  with 
actual  practical  knowledge  of  se  ’"etarial  work.  Which  one 
do  you  think  it  is? 

3.  Which  writer  would  be  the  most  helpful  source  of  advice  to  a 
beginning  secretary? 

5.  Results  and  Implications 

The  results  were  encouiaging,  even  though  the  small  number  of 
subjects  in  the  population,  and  the  characteristics  of  the  investigation, 
limited  considerably  the  statistical  significance  of  the  results.  However, 
the  logical  consistency  of  the  approach  and  the  realization  that  it  was 
possible  to  write  sensible  messages  on  the  ba^4s  of  the  data  obtained 
indicated  the  feasibility  of  the  basic  idea  and  the  possibility  that  the 
method  could  be  developed  into  i:  useful  tool. 

During  this  test,  it  was  realized  that  in  order  to  have  an  unbiased 
selecticr.  of  categories  and  to  make  full  use  of  the  information  contained 
in  the  data  it  should  be  classified  by  members  of  the  same  population. 

This  prompted  the  second  study,  which  was  made  with  students  at  University 
of  California,  Santa  Barbara  (UCSB) . 
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B.  HIGH  SCHOOL  AND  COLLEGE  STUDENTS 

1 .  Sub j  ec  ts  ( Ss ) 

One  hundred  students  enrolled  in  an  introductory  sociology  course 
during  th?  1966  summer  session  at  UCSB  served  as  Ss.  They  were  divided 
into  two  equal  groups  (ECHO  I  and  ECHO  I'),  each  group  consisting  of 
about  35  high  school  juniors  and  about  15  college  students.  Both  groups 
received  the  same  treatment  except  that  the  Echo  I'  group  was  tested  one 
week  after  the  Echo  I  group. 

1.  Data  Collection  Procedure 

The  ECHO  I  subjects  were  divided  into  two  equal  sections,  both 
sections  receiving  identical  instructions  and  treatment  except  that  the 
order  in  which  the  questions  were  presented  was  reversed  for  the  two 
sections  to  control  for  order  effect. 

Each  was  given  ten  blue  and  ten  brown  IBM  cards.  They  were  instructed 
to  indicate  their  academic  status  by  printing  an  "H"  ("high  school  student") 
or  an  "0"  ("other")  in  the  lower  left-hand  corner  of  each  card.  They  were 
asked  to  write  legibly  and,  beginning  with  the  brown  cards,  to  answer  the 
following  question  (negative  condition) : 

For  a  person  like  yourself,  in  his  everyday  life,  give 
a  specific  exarnpl?.  of  something  important  that  could  happen, 
chat  would  be  an  unfortumate .  unfavorable  and  undesirable 
thing  to  have  happen,  and  that,  if  it  happened  to  you,  you 
would  be  unhappy . 

This  question  was  first  read  aloud,  then  written  on  the  blackboard, 
and  finally  restated  in  a  less  formal  manner. 

After  the  students  had  written  their  first  answer  to  this  question, 
they  were  instructed  to  print  (in  the  lower  right-hand  corner  of  the  card) 
who  or  what  would  be  wholly  or  mostly  responsible  for  this  event.  At 
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this  point  the  students  were  told  that  they  would  have  ten  minutes  to 
complete  as  many  of  the  brown  ("negative")  cards  as  they  could. 

After  the  students  had  wc.ked  for  10  minutes  on  the  brown  cards, 
they  were  instructed  to  put  them  aside  and  to  take  out  the  blue  cards. 
The  same  procedure  was  employed  with  a  different  question: 


For  a  person  like  yourself,  in  his  everyday  life,  give  a 
specific  example  of  something  important  that  could  happen 
that  would  be  a  fortunate,  favorable  and  desirable  thing 
to  have  happen  and  that,  if  it  happened  to  you,  you  would 
be  pleased. 


Again,  the  students  were  instructed  to  indicate  the  "source,"  i.e.,  the 
person  who  would  be  mainly  responsible  for  this  thing  happening.  They 
were  given  10  minutes  to  complete  as  many  blue  cards  as  they  could. 


The  students  were  then  instructed  to  put  the  blue  cards  aside  and 
to  rate  the  brown  cards  in  terms  of  their  importance  using  the  "I"  box 
(upper  left-hand  corner  of  the  card).  The  instructions  were  as  follows: 

If  you  had  one  'wish  and  you  could  make  sure  that  one  of 
these  things  would  never  come  true... then  write  in  the  upper 
left-hand  corner  of  the  card  the  number  1.  Now  look  at  the 
remaining  cards  and  do  the  same  thing.  The  biggest  number 
will  then  be  the  one  thing  that  you  would  allow  to  happen  if 
one  thing  had  to  happen. 

In  this  manner,  the  subjects  assigned  importance  numbers  to  first  the 
brown  cuds  and  then  the  blue  cards. 


The  final  task  was  the  assignment  of  an  "uncertainty"  or  probability 
(P)  estimate  to  the  cards.  The  instructions  were  as  follows: 

Now  take  the  brown  cards  again  and  spread  them  out.  Ignore  the 
number  you  have  put  in  the  "I"  box.  Look  at  the  answers  and, 
taking  them  in  any  order,  ask  yourself  the  following  thing 
about  the  event  you  have  specified:  "How  likely  is  this  to 
come  to  be  true?"  Think  of  it  in  this  way:  If  a  hundred  people 
like  yourself  all  had  the  desire  for  this  outcome,  how  many 
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do  you  think  would  get  their  wish?  Write  that  number  in 
the  upper  right-hand  corner  of  the  card. 

In  this  manner,  the  subjects  assigned  uncertainty  values  to  all  of  the 
events.  Similar  instructions  were  given  for  the  blue  cards. 

Each  card  was  completed  as  shown  in  Fig.  5. 


I  = 

P  = 

SPECIFIC  EVENT 

H  or  0 

SOURCE 

Figure  5.  Data  Collection  Card  Used  in  ECHO  I  Study 


3.  Classification  Procedure 

Three  persons  indigenous  to  the  subject  population  (i.e. ,  students 
from  the  class)  served  as  classifiers  and  sorted  the  responses  in  the 
following  manner : 

1.  Each  classifier  was  given  one-third  of  the  blue  IBM  cards 
(positive  responses)  and  instructed  to  put  together  those 
cards  which  had  the  same  meaning. 

2.  The  first  classifier  to  complete  this  initial  sorting  made 
labels  for  the  categories  he  had  created.  When  the  others 
i.id  finished  their  sorting,  the  first  classifier  read  each 
of  his  labels  aloud  and  the  others  handed  him  their  cards 
which  they  believed  fell  into  the  same  category.  In  this 
manner,  the  three  separate  classification  systems  were  com¬ 
bined  to  make  one. 
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3.  Each  category  was  then  discussed  individually.  The  cards 
were  read  aloud  and  the  classifiers  were  asked  to  decide 
which  of  the  responses  actually  belonged  in  the  category. 

When  it  seemed  appropriate,  categories  were  broken  down 
into  smaller  categories.  The  resulting  classification 
system  consisted  of  15-20  categories  describing  the  subjects' 
responses  to  the  positive  question.  The  brown  cards,  i.e., 
responses  to  the  negative  question,  were  subsequently  classi¬ 
fied  in  the  same  manner  by  the  same  procedure. 

4 .  Results  and  Implications 

No  message  session  was  employed  because  this  study  was  designed 
to  examine  the  relationships  between  the  subjects'  expectations  and  the 
uncertainty  and  importance  they  attached  to  those  attitudes.  Several 
mathematical  relationships  were  proposed  but  the  ordinal  nature  of  the 
importance  and  uncertainty  measures  prohibited  useful  mathematical  manipu¬ 
lation  of  the  variables. 

However,  the  results  did  show  that,  in  principle,  a  classification 
method  employing  indigenous  classifiers  could  be  developed  in  such  a  way 
as  to  fully  use  the  information  collected  via  the  ECHO  methodology. 

In  addition,  ECHO  appeared  to  be  an  effective  tool  for  discovering 
and  analyzing  the  different  systems  of  social  control  of  the  various 
subgroups  within  the  subject  culture. 

C.  DEVELOPMENT  OF  ECHO  DATA  COLLECTION  INSTRUMENT 

1.  Materials 

IBM  cards  were  chosen  as  the  data  collection  instrument  because 
of  easy  manipulation  in  the  classification  session,  efficient  sorting 
and  later  retrieval  of  information,  and  direct  access  to  computer  opera¬ 
tions  without  transferring  data.  Data  collection  packets  were  made  up 
as  follows: 
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1.  Twenty  IBM  cards  were  presented  to  each  subject:  10  were 
printed  with  the  positive  questions:  "What  is  a  good  thing 

to  do?  Who  would  approve?"  and  10  with  the  negative  questions 
"What  is  a  bad  thing  to  do?  Who  would  disapprove?" 

2.  The  cards  were  prepunched  with  numbers  identifying  the  subject 
the  population  from  which  he  was  drawn,  the  valence  of  the 
assigned  evaluation  (positive  or  negative),  and  the  sequence 
in  which  the  wrote  his  responses. 

3.  Ranking  instruction  cards  were  included  in  the  packets, 
instructing  the  Ss  to  rank  their  responses  to  each  question 
from  1  to  10  in  order  of  decreasing  importance. 

4.  Role  instruction  cards,  placed  first  in  the  packets,  instruc¬ 
ted  the  subjects  to  assume  specific  roles  in  answering  the 
ECHO  questions. 

5.  Biographical  information  cards .  placed  last  in  the  packets, 
requested  information  about  the  S/ s  age,  sex,  marital  status, 
academic  status,  and  length  of  attendance  at  the  school.. 

2 .  Application  of  ECHO  Data  Collection  Instrument — UCLA  Pilot  Study 

a.  Data  Collection  Conditions 

(1)  Role  Assignments 

Each  person  may  play  several  different  roles  concurrently.  For 
example,  a  man  might  be  a  student,  a  son,  a  husband,  and  an  employee  at 
the  same  time.  He  will  have  different  systems  or  hierarchies  of  values 
and  attitudes  to  correspond  with  these  different  roles.  For  example, 
in  his  role  as  a  husband,  attitudes  and  actions  related  to  his  rela¬ 
tionship  with  his  wife  might  be  of  greatest  importance  while,  in  his  role 
as  a  student,  studying  and  getting  good  grades  will  predominate.  There¬ 
fore,  in  trying  to  assess  the  person's  value  system,  it  is  necessary  to 
aefine  the  context,  or  role,  in  terms  of  which  he  is  to  respond;  that 
is,  to  indicate  the  value  system  and  its  attendant  role  in  which  we  are 
interested . 


48 


UNCLASSIFIED 


UNCLASSIFIED 


The  UCLA  subjects  were  studied  in  terms  of  two  different  roles: 
some  of  the  subjects  were  instructed,  via  printed  role  instruction  cards 
placed  first  in  their  packets,  to  answer  the  ECHO  questions  in  terms  of 
their  roles  as  "students  at  UCLA";  others  were  given  no  specific  role 
instructions  and  hence  could  assume  any  role  they  desired.  This  second 
condition  was  termed  the  "general  person  role". 

(2)  Sequential  Conditions 

To  control  for  any  possible  sequential  effects  which  might  result 
from  the  order  in  which  the  positive  and  negative  questions  were  answered, 
the  Ss  were  divided  into  two  groups.  Group  A  answered  the  10  positive 
questions  first  and  the  10  negative  questions  second;  Group  B  the  reverse. 

(3)  Ranking  Conditions 

All  the  subjects  were  instructed  to  rank  their  responses  in  order 
of  importance.  Three  ranking  conditions  were  employed  in  an  effort  to 
determine  at  what  place  in  the  tests  the  subjects  should  rank  their 
responses.  The  Ss  under  ranking  condition  I  completed  the  cards  in  deck  1, 
completed  the  cards  in  deck  2,  ranked  deck  1,  and  ranked  deck  2.  Ranking 
condition  II  Ss  completed  deck  1,  ranked  deck  1,  completed  deck  2,  and 

ranked  deck  2.  For  ranking  condition  III,  positive  and  negative  cards 
were  intermixed  in  decks  1  and  2.  These  subjects  completed  deck  1, 
completed  deck  2,  ranked  deck  1,  and  ranked  deck  2.  The  way  in  which 
the  Ss  were  grouped  by  role,  sequential  condition,  and  ranking  condition, 
is  shown  in  Table  8. 

TABLE  8 

DESIGN  OF  DATA  COLLECTION  CONDITIONS— UCLA  PILOT  STUDY 


Ranking  Condition 


Role  Condition 

I 

II 

III 

Student 

A(N  =  20) 

A(N  -  20) 

N  "  V 

B(N  -  20) 

B(N  -  20) 

General  Person 

A(N  -  20) 

B(N  -  20) 
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b.  Subjects 

One  hundred  thirty-seven  students  enrolled  in  an  undergraduate  psy¬ 
chology  course  at  UCLA  served  as  subjects.  The  subjects  were  predominately 
upper  classmen  (juniors  and  seniors)  and  their  ages  ranged  from  18  to  47 
years. 


c.  Method 

(1)  Mat  s 

Each  subjc-  eceived  a  packet  consisting  of  20  printed,  prepunched 
IBM  cards,  2  ranking  instruction  cards,  and  a  biographical  information 
card.  The  subjects  under  the  "student  role"  condition  also  received  role 
instruction  cards. 

(2)  Procedure 

After  the  packets  had  been  distributed  and  the  instructions  read 
by  the  investigator,  the  subjects  were  permitted  to  complete  their 
packets  at  their  own  speed.  The  instructions  to  the  subjects,  which  were 
read  aloud  by  the  investigator,  were  as  follows: 

INSTRUCTIONS  TO  THE  SUBJECTS 


The  purpose  of  this  study  is  to  develop  methods  for 
improving  communications  between  different  cultures.  Any 
such  interculture  communication  is  necessarily  based  on  an 
understanding  of  the  value  systems  inherent  in  each  culture. 

We  are  currently  investigating  an  instrument  with  which  we  hope 
we  will  be  able  to  discover  and  examine  the  value  system  for 
any  cultures.  We  plan  to  test  the  effectiveness  of  our  metho¬ 
dology  by  comparing  the  culture  which  has  developed  in  this 
group  with  a  somewhat  similar  culture  which  has  developed 
within  another  group  of  students. 

Each  of  you  should  have  a  sealed  envelope  and  a  pen. 

These  are  the  only  materials  you  will  need.  You  will  be  asked 
to  give  ten  responses  to  each  of  the  following  questions:  "What 
do  you  think  a  person,  like  yourself,  could  do  that  would  be 
a  good  thing  to  do  and  that  someone  would  approve  of  your 
doing?"  "Who  would  approve  of  your  action?"  Likewise,  "What 
do  you  think  a  person,  like  yourself,  could  do  that  would  be  a 
bad  thing  to  do  and  that  someone  would  disapprove  of  your 
doing?"  "Who  would  disapprove?" 
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When  we  tell  you  to  begin,  you  are  to  open  the  envelope 
by  tapping  it  down  on  the  left  side  so  that  the  cards  within 
are  down  on  the  left.  Then  tear  off  the  right  edge.  Take  out 
the  deck  of  cards  which  is  numbered  with  a  "1".  Do  not  remove 
anything  else  from  the  envelope.  Take  the  rubber  band  off  the 
cards  but  be  especially  careful  that  you  do  not  change  the 
order  of  the  cards.  Writing  or  printing  as  neatly  as  possible 
give  a  specific  example  of  a  good  or  bad  thing  to  do,  accord¬ 
ing  to  the  question  which  is  printed  on  the  card.  Write  that 
answer  on  the  lines  which  have  been  provided  following  the 
question.  Then  write  the  title  or  position  of  the  person  who 
would  approve  or  disapprove  of  your  action  on  the  lines  which 
have  been  provided  following  that  question.  When  you  have 
answered  all  ten  cards  in  deck  1,  put  them  back  into  the 
envelope  and  take  out  deck  "2”.  Again,  we  ask  you  not  to  change 
the  order  of  the  cards.  Follow  the  same  procedure  as  before 
in  writing  your  answers  on  these  cards.  When  you  are  finished, 
return  these  cards  to  the  envelope  and  take  out  the  yellow 
"Biographical  Information"  card.  Answer  each  of  the  questions 
on  this  card  by  placing  an  "x"  in  the  appropriate  box.  We 
wish  to  thank  you  all  again  for  donating  your  time  to  help  make 
this  study  a  success. 

Do  you  have  any  questions?  If  a  problem  should  arise  whiJe 
you  are  working,  please  raise  your  hand.  You  may  begin. 


In  response  to  requests  for  clarification,  the  investigator  gave 
the  Ss  the  following  example:  "If  a  fireman  were  asked  'What  is  a  good 
thing  to  ao?'  and  'Who  would  approve?',  he  might  answer,  'A  good  thing  to 
do  is  to  keep  my  fire  engine  in  good  running  condition  and  the  fire  chief 
would  approve  of  this  act.'"  In  addition,  the  subjects  asked  whether  or 
not  they  could  list  themselves  as  sources  of  approval  or  disapproval. 

The  investigator  answered  that  they  could  indeed  list  themselves  or  anv- 
one  else  as  sources. 


3 .  Results  of  UCLA  Pilot  Study  of  Data  Collection  Instrument 

a.  Evaluation  of  Data  Collection  Instrument 

The  data  collection  packet,  consisting  of  printed,  prepunched  IBM 
cards,  is  an  efficient  means  of  collecting  responses  to  the  ECHO  questions, 
fhe  Ss  easily  handled  the  20  cards,  responding  to  all  of  them  in  20-30 


UNCLASSIFIED 


51 


UNCLASSIFIED 


minutes.  No  sequential  effect  arising  from  the  order  of  presentation  of 
positive  and  negative  evaluations  was  discovered. 

However,  an  ordinal  effect  was  found  relating  the  order  in  which 
specific  responses  were  written  and  the  importance  estimates  assigned  by 
the  subject  to  those  responses.  In  general,  the  subjects  tended  to  write 
the  most  important  responses  first.  See  Appendix  II. 

The  instructions  to  the  subjects  proved  to  be  adequate  in  that 
almost  all  subjects  correctly  followed  the  prescribed  procedure.  How¬ 
ever,  two  changes  were  made  in  the  instruction  for  future  studies: 


1.  No  examples  of  the  desired  responses  will  be  given  as  any 
such  example  might  bias  the  Ss'  responses. 

2.  The  instructions  will  indicate  that  the  Ss  may  list  themselves 
as  sources  oi.  approval  or  disapproval  if  they  wish. 


b.  Data  Collection  Sessions  at  Carnation  and  Prudential 
(1)  Subjects 

These  two  groups,  studied  concurrently,  consisted  of  clerical  employ¬ 
ees.  The  compositions  of  the  two  groups  differed  slightly.  The  54  Carna¬ 
tion  subjects  (39  secretaries  and  15  clerk-typists)  on  the  average,  were 
older  and  had  more  education  than  the  Prudential  subjects.  In  addition, 
they  were  more  homogeneous,  in  terms  of  age,  education,  and  length  of 
employment,  than  the  Prudential  subjects.  The  Prudential  subject  population 
consisted  of  women  employed  at  Company  job  levels  3,  4  and  5,  the  majority 
working  as  clerks.  Of  this  group,  93  were  chosen  randomly  to  serve  in  the 
data  col'cction  phase  of  the  study. 
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(2)  Data  Collection  . rocedure 

(a)  Materials 

Each  data  collection  packet  consisted  of  20  IBM  cards.  Ten  of  the 
card.;  had  printed  on  them:  "What  is  a  good  thing  to  do?'1  "Who  would 
approve?"  and  the  other  ten  cards  read:  "What  is  a  bad  thing  to  do?" 

"Who  would  disapprove?"  Each  card  was  prepunched  with  group  and  subject 
identification  numbers  and  a  number  to  identify  the  sequence  in  which  the 
cards  were  filled  out.  Ranking  instruction  and  biographical  information 
cards  were  also  included  in  the  packets.  The  former  instructed  the 
subject  to  rank  the  cards  in  each  deck  in  the  order  of  decreasing  impor¬ 
tance.  The  latter  elicited  information  regarding  ".he  subject's  age, 
sex,  education,  length  of  employment,  and  job  title. 

(b)  Procedure 

The  packets  were  distributed  and  the  subjects  instructed  to  work 
at  their  own  speed  in  answering  the  questions  printed  on  the  cards. 

All  the  subjects,  with  the  exception  of  12  Carnation  subjects, 
were  instructed  to  list  good  and  bad  things  to  do  5n  terms  of  their  roles 
as  employees.  The  12  Carnation  subjects  were  instructed  verbally  to  ans¬ 
wer  the  questions  in  terms  of  their  roles  as  persons,  i.e.,  in  terms  of 
their  roles  in  private  life,  rather  than  restricting  themselves  to  work- 
oriented  responses. 

The  following  instructions  were  read  to  the  subjects. 

INSTRUCTIONS  TO  THE  SUBJECTS 


Before  we  begin,  we  want  to  assure  you  that  your  answers 
will  be  considered  confidential  and  that  no  company  personnel 
will  have  access  to  them.  In  addition,  your  anonymity  is 
assured  as  there  is  no  possible  way  in  which  we  could  identify 
any  of  you  from  your  answers.  We  wish  to  emphasize  that  these 
questions  are  not  designed  as  tests  of  your  ability  or  intelli¬ 
gence.  Also,  there  are  no  right  or  wrong  answers  as  we  merely 
want  vour  opinion. 
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Each  o£  you  has  a  sealed  envelope  and  something  to  write 
with.  These  are  the  only  materials  you  will  need  this  evening. 

You  will  be  required  to  list  ten  responses  to  ?ach  of  the 
following  questions:  These  questions  are:  "What  do  you  think  a 
person  like  yourself,  in  your  posJtion  at  this  company,  could 
do  that  would  be  a  good  thing  to  do  and  that  someone  would 
approve  of  your  doing?"  and  "What  is  the  job  title  or  position 
(NOT  THE  NAME)  of  the  person  who  would  approve  of  your  doing 
that  thing?"  By  this,  we  mean — who  would  be  aware  that  you  had 
done  this  and  who  would  be  "pleased"?  "What  do  you  think  a 
person  like  yourself  in  your  position  at  this  company  could  do 
that  would  be  a  bad  thing  to  do  and  that  someone  would  disapprove 
of  your  doing?"  and  "What  is  the  job  title  or  position  of  the 
person  who  would  disapprove  of  your  doing  that  thing?"  For  ex¬ 
ample,  the  person  who  approves  or  disapproves  might  be  your  friends, 
your  boss,  employer,  supervisor,  your  mother,  or  yourself. 


Open  the  envelope  by  tapping  it  down  on  the  left  3ide  so 
that  the  cards  within  it  are  down  on  the  left.  Then  tear  off 
the  right  edge.  Take  out  the  deck  of  cards  which  is  numbered 
with  a  "1".  Do  not  remove  anything  else  from  the  envelope. 

Take  the  rubber  band  off  the  cards  but  be  especially  careful 
that  you  do  not  cha  ge  the  order  of  the  cards.  Writing  or 
printing  as  neatly  as  possible  give  a  specific  example  of  a 
good  thing  or  a  bad  thing  to  do,  according  to  the  question  which 
is  printed  on  the  card.  Write  that  answer  on  the  lines  which 
have  been  provided  following  the  question.  Then  write  the  title 
or  position  of  the  person  who  would  approve  or  iisapprove  of  your 
action  on  the  lines  which  have  been  provided  following  that 
question.  When  you  have  answered  all  ten  cards  in  deck  1,  put 
the  rubber  band  back  around  the  cards ,  making  sure  that  the 
piece  of  paper  with  the  large  number  "1"  is  attached  to  them, 
and  out  the  cards  aside.  Then  take  the  deck  of  cards  with  the 
large  number  "2"  out  of  the  envelope  and  remove  the  rubber  band. 
Again,  we  ask  you  not  to  change  the  order  of  the  cards.  Follow 
the  same  procedure  as  before  in  writing  your  answers  on  these 
cards.  When  you  are  done,  replace  the  rubber  band  and  the  "2" 
and  put  the  cards  aside.  When  you  have  finished  answering  the 
cards  in  deck  "2"  you  will  be  ready  to  begin  the  next  step  in 
this  survey.  You  will  find  two  white  cards  in  your  envelope. 
These  cards  will  instruct  you  to  rank  your  answers  in  terms  of 
!.-•> w  important  each  answer  is  to  you.  Rank  deck  "1"  first. 

You  can  do  this  in  the  following  manner.  Spread  the  cards  out 
on  the  table  in  front  of  you  so  that  you  can  read  the  ten 
actions  you  wrote.  Pick  the  action  which  you  think  is  the 
most  important  and  put  a  "1"  in  the  box  in  the  lower  left- 
hand  corner  of  the  card.  Now,  find  the  action  which  is  second 
in  importmee  to  you  and  put  a  "2"  in  the  box. 
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You  will  put  a  "3"  in  the  box  on  the  card  on  which  you 
wrote  the  action  which  you  think  is  third  in  importance. 
Continue  in  this  manner  until  you  have  ranked  all  ten  cards. 
When  you  are  done,  you  will  have  numbered  the  cards  from  1 
to  10  with  a  "1"  on  the  most  important  card  and  a  "10"  on  the 
least  important  card.  Please  recheck  the  cards  to  make  sure 
that  r  ■>  ctrds  have  the  same  number  on  them.  When  you  are 
finis  :iJ,  replace  the  rubber  band  and  paper  numbered  with  the 
large  "1"  and  put  these  cards  aside.  Then,  you  are  to  rank 
the  ten  cards  in  deck  "2"  in  the  same  manner.  After  you 
have  ranked  both  decks  of  cards  take  the  yellow  card  out  of 
the  envelope.  Answer  each  of  the  questions  on  the  Biographi¬ 
cal  Information  card  by  placing  an  "x"  in  the  appropriate 
box.  This  is  the  last  part  r2  tonight's  session.  When  you 
are  finished,  put  all  the  cards  back  in  t’ e  envelope  and 
bring  them  down  to  the  front  of  the  room. 

Do  you  have  any  questions  at  this  time?  If  any  problems 
should  arise  while  you  are  working,  please  raise  your  hand. 

You  may  begin. 


D.  DEVELOPMENT  OF  DATA  CLASS IF  NATION  METHOD— UCLA  PILOT  STUDY 


1.  Classification  Teams 

Two  teams  of  three  personR  each  were  chosen  from  the  subject  popula¬ 
tion  to  serve  as  indigenous  classifiers.  One  team  consisted  of  two  males 
and  one  female  while  the  other  team  was  made  up  of  two  females  and  one 
male. 


2 .  Procedure 


a.  Initial  Sorting 

One  team  of  three  classifiers  received  1143  positive  cards  while  the 
other  team  received  1088  negative  cards. 

Each  classifier  (C)  was  given  one-third  of  the  cards  to  be  classi¬ 
fied  by  his  team  and  was  Instructed  to  read  all  the  cards  and,  then,  to 
put  together  those  cards  which  had  the  same  meaning.  This  initial  sorting 
resulted  in  several  small  stacks  of  cards  representing  different  types 
of  responses  or  categories. 
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b.  Creation  of  Larger  Categories 

The  members  of  the  classification  teams  worked  together  in  this 
phase  of  the  session.  Taking  turns,  each  C  picked  up  one  of  his  stacks 
of  cards  and  read  the  cards.  The  other  Cs  then  hanued  him  al]  of  their 
cards  which  they  felt  belonged  in  the  category  under  discussion.  In  this 

manner  the  three  separate  sets  of  categories  createu  in  the  initial 

sorting  were  combined  to  form  one  classification  system. 

c.  Refinement  of  Categories 

The  Cs  discussed  each  individual  category,  reading  ail  of  the 
cards,  and  occasionally  divided  large  categories  into  smaller  ones. 

d.  Assignment  of  Category  Labels 

The  £s  labeled  Lhe  categories,  creating  titles  which,  in  the  form 
of  a  response  to  the  original  ECHO  question,  would  represent  all  of  the 
res. onses  within  each  category. 

e.  Estimation  of  Category  Importance 

The  C's  final  task  was  to  rank  the  categories  from  1  to  N,  where 
N  equals  the  number  of  categories,  in  order  of  decreasing  importance. 

3.  Evaluation  of  Data  Classification — UCLA  Pilot  Study 

The  method  itself  was  workable  and  seemed  logical  in  that  the  classi¬ 
fication  progresses  from  the  creation  of  small,  exclusive  stacks  of  cards 
to  larger,  more  general  categories. 

The  use  of  three  classifiers  per  team  appeared  to  be  an  efficient 
arrangement.  Small-groups  research  has  proven  that  triads  are  superior 
to  dyads  in  making  decisions  f.ince  there  is  a  majority  when  an  odd  number 
of  persons  are  employed.  Three  also  seems  a  gcod  number  as  any  more,  say 
five,  would  have  made  the  group  too  big,  resulting  in  less  direct  inter¬ 
action  among  the  members. 
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The  classification  session,  lasting  roughly  12  hours  because  of 
the  large  number  of  cards,  was  clearly  too  long.  The  team  which  sorted 
the  positive  responses  created  69  categories  from  the  1143  cards  while 
the  other  team  created  71  categories  from  the  1088  negative  cards. 

The  length  of  the  session,  the  enormousness  of  the  task  Itself,  and 
the  lac«v  of  regular  breaks  led  to  the  development  of  extreme  fatigue  and 
boredom  among  the  classifiers.  Therefore,  the  reliability  of  the  result¬ 
ant  classification  systems  was  questioned. 

In  addition,  the  classifiers  created  such  a  large  number  of  cate¬ 
gories  that  the  categories  themselves  represented  specific  actions  rather 
than  generalized  attitudes  and  hence,  were  too  specific  and  detailed  to 
represent  the  more  generalized  attitudes  of  the  subject  population.  The 
multiple  classification  process  discussed  later  would  have  resolved  this 
problem.  No  conclusions  were  reached  on  the  efficacy  of  classifying 
positive  and  negative  responses  together  or  separately. 


4*  Baja  Classification  Sessions  at  Carnation  and  Prudential 

a,  Classification  Teams 

Two  teams  (CA  and  CB)  of  three  Cs  each  were  employed  to  classify  the 
1042  responses  from  Carnation.  Two  teams  ^  and  P3>  of  3  Cs  and  one 
team  ( ?2 )  of  2  Cs,  due  to  the  unexpected  absence  of  one  classifier,  were 
employed  to  sort  the  1671  cards  from  Prudential.  The  classification 
sessions  for  the  two  different  subject  groups  were  held  concurrently. 

b.  Distribution  of  Cards 

Carnation.  Classification  * =am  CA  received  516  positive  and 

negative  cards  while  team  CP  classified  526  positive  and  negative 
cards. 

Prudential.  Classification  team  P^  sorted  599  positive  cards; 

team  ?2  sorted  472  negative  cards;  and  team  P3  sorted  600  positive 
and  negative  cards. 


UNCLASSIFIED 


57 


UNCLASSIFIED 


c.  Pro  edure 

(1)  Initial  Sorting 

As  in  the  UCLA  classification  session,  the  classification  team  mem¬ 
bers  worked  independently  in  their  initial  sorting  of  the  cards. 

(2)  Creation  of  Larger  Categories 

The  team  members  were  brought  together  for  this  phase  in  which 
the  individual  classifiers'  stacks  of  cards  were  combined  to  form  larger 
categories . 

The  investigators  circulated  among  the  various  classification  teams 
to  prevent  any  one  member  from  dominating  his  team's  discussions  and  to 
keep  the  session  moving  smoothly. 

(3)  Refinement  of  Catego-  s 

Taking  turns,  the  £s  read  all  of  the  carls  in  each  category.  Where 
they  felt  it  was  necessary,  the  Cs  refined  a  category  by  omitting  or 
adding  cards  or  breaking  it  down  into  two  smaller  categories.  The  f<nal 
classification  systems  contained  the  following  numbers  of  categoric 
Carnation — team  CA  created  28  categories  and  te-‘.m  CB  created  30;  Pruden¬ 
tial-team  P^  created  22  categories,  team  created  29  categories,  and 
team  created  20  categories. 

(4)  Assignment  of  Category  Labels 

The  Cs  labeled  the  categories,  creating  titles  which,  in  the  form 
of  a  response  to  the  original  ECHO  question,  represented  all  of  the 
responses  within  each  category.  As  these  labels  were  general,  the  Cs 
also  listed  actions  and  attitudes  as  examples  of  the  behavior  represented 
by  the  category  titles. 

(5)  Estimation  of  Category  Importance 

The  Cs  ranked  their  categories  from  1  to  N  (where  N  equals  the 
total  number  of  categories)  in  order  of  decreasing  importance.  As  a  test 
of  importance  ranking  reliability,  Carnation's  team  CA  ranked  by  impor¬ 
tance  the  categories  create  1  by  team  CB.  The  Spearman's  Rank-Difference 
correlation  between  the  two  rankings  of  the  CL  categories  resulted  in  a 
significant  Spearman  rho  »  .79,  a  ■  .01*. 
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Both  teams  of  Carnation  classifiers  ranked  as  most  important 
the  categories  created  from  responses  of  1<  Carnation  data  collection 
subjects  who  were  assigned  "general  person  roles"  rather  than  being 
restricted,  as  were  all  of  the  other  Ss,  to  work-oriented  responses • 

This  result  led  to  some  discussion  regarding  the  assignment  of  roles. 

It  was  felt  that,  perhaps,  the  most  important  attitudes  would  be  elicited 
if  the  Ss  were  allowed  to  assume  the  roles  which  they,  themselves,  con¬ 
sidered  the  most  important  parts  of  their  lives. 


E.  APPLICATION  OF  ECHO  METHODOLOGY  AT  STANFORD 

1.  Subjects 

Seventy-one  men  and  1  woman  enrolled  in  the  Graduate  School  of 

Business  at  Stanford  University  served  as  subjects.  The  subjects  ranged 
in  age  from  22  to  AO  years. 

2.  Data  Collection  Procedure 

a  Materials 

Lata  collection  packets  identical  to  those  employed  in  the  UCLA 
pilot  study  were  prepared. 

b.  Data  Collection  Conditions 

(1)  Role  Assignments 

Half  of  the  Ss  were  instructed  by  role  instruction  cards  to  assume 
roles  as  'students  at  Stanford"  in  completing  their  packets.  The  others 
were  instructed  to  assume  their  more  general  roles  as  "persons  in  society". 

(2)  Sequential  Conditions 

Group  A  Ss  received  the  10  positive  cards  first  and  the  10  negative 
cards  last.  The  reverse  war,  true  for  the  Group  B  Ss .  The  distribution 
among  the  Ss  of  the  role  assignments  and  sequential  conditions  is  shown 
in  Table  9. 
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(3)  Ranking  Conditions 

All  ch.  Ss  were  instructed  to  write  deck  1,  rank  deck  1,  write 
deck  2,  and  rank  deck  2.  (This  is  identical  to  ranking  condition  II 
employed  in  the  UCLA  study).  The  use  of  this  seguence  was  based  on  the 
aesu-ption  that  the  Ss  would  be  best  able  to  compare  and  rank  their 
responses  immediately  after  writing  them. 


TABLE  9 

DESICN  OF  DATA  COLLECTION  CONDITIONS — STANFORD 


Sequential  Control 


Role  Condition 

A<+,  -) 

B(~,  +) 

Tota. 

Student  Role 

N  =  18 

N  =  18 

36 

General  Person 

N  =  18 

N  =  18 

36 

Total 

36 

1 - - - 

36 

72 

c.  Procedure 

After  the  data  collection  packets  had  been  distributed,  the  instruc¬ 
tions  were  read  aloud  to  the  subjects. 


INSTRUCTIONS  TO  THE  SUBJECTS 


.  .  6  PurPose  of  this  study  is  to  develop  methods  for 

improving  communications  between  different  cultures.  Any  such 

stfleH7CUlt^r\C0MnUniCati°n  iS  necessarily  based  on  an  under¬ 
standing  of  the  value  systems  inherent  in  each  culture.  We 

weewilirbeCininrS^8atin8  3n  instrument  with  which  we  hope 
will  be  able  to  discover  and  examine  the  value  syste™  for 

dolocv  JUre*  We.plan,t0  Lest  the  effectiveness  of  our  metho¬ 
dology  by  comparing  the  culture  which  nas  developed  in  this 
class  with  a  somewhat  similar  culture  which  has  developed 
within  a  class  of  students  at  UCLA.  P 


Each  of  you  should  have 
These  are  the  only  materials 
to  give  ten  responses  to  each 


a  sealed  envelope  and  a  pen. 
you  will  need.  You  will  be  asked 
of  the  following  questions: 
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"What  do  you  think  a  person,  like  yourself,  could  do  that  would 
be  a  good  thing  to  do  and  that  someone  would  approve  of  your 
doing?"  "What  is  the  position,  with  respect  to  yourself,  of 
the  person  who  would  approve  of  your  action?"  Likewise,  "Wk.at 
do  you  think  a  person,  like  yourself,  could  do  that  would  be 
a  bad  thing  to  do  l  ig  that  someone  would  disapprove  of  your 
doing?"  "What  is  the  position,  with  respect  to  yourself,  of 
the  person  who  would  disapprove?" 

You  will  write  your  responses  on  the  IBM  cards  which  are 
in  the  envelope.  You  are  to  open  the  envelope  by  tapping  it 
down  so  that  the  cards  within  it  are  down  at  the  left  end  of 
the  envelope.  Then  tear  off  the  right  edge  of  the  envelope, 
forming  a  packet  for  the  cards . 

After  you  have  read  the  special  instruction  card  which 
>  is  the  first  card  in  the  packet,  take  out  the  deck  of  cards 
which  has  been  marked  with  a  number  "1".  Be  especially  care¬ 
ful  that  you  do  not  disturb  the  order  of  these  card  while 
you  answer  the  questions  or.  them.  When  you  have  answered  all 
ten  cards,  f^low  the  instructions  at  the  back  of  deck  "1". 
After  you  have  finished,  replace  the  rubber  band  around  deck 
"1"  and  go  on  to  deck  "2",  following  the  same  procedure. 

Upon  completion  of  deck  "2",  fill  out  the  biographical 
data  card  and  then  put  all  the  materials  back  into  the  packet. 

Bring  the  packet  to  the  front  of  the  room  and  hand  it  to 
one  of  us  as  you  leave.  We  wish  to  thank  you  for  your  coopera¬ 
tion.  Are  there  any  questions?  You  may  begin. 


3.  Data  Classification  Session 

a.  Classification  Teams 

Two  classification  teams,  each  consisting  of  three  men  selected 
from  the  subject  population,  were  employed  in  the  classification  session. 

b.  Distribution  of  Cards 

One  team  sorted  the  710  positive  responses  while  the  other  team 
sorted  the  negative  responses. 


C.  Procedure 

The  data  classification  procedure  was  identical  to  that  employed 
at  UCLA,  progressing  from  the  initial  sorting  to  the  creation  of  larger 
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categories  to  the  refinement  and  labelir,  of  the  categories  and,  finally, 
to  the  ranking  of  the  ca.  egories  in  terms  of  importance.  The  resultant 
classification  systems  consisted  of  29  positive  categories  and  31  negative 
categories . 

F.  DEVELOPMENT  OF  .  VALIDI”^  TEST— UCLA  PILOT  STUDY 

1.  Purpose  of  Validity  Test 

The  validity  test,  or  "message  session",  was  designed  as  an  evalua¬ 
tion  of  the  effectiveness  of  the  ECHO  data  collection  instrument  in  deter¬ 
mining  the  prevalent  and  important  attitudes  within  the  subject  culture's 
value  system.  Subjects  were  instructed  to  compare  and  evaluate  actions 
and  attitudes  represented  by  lists  of  category  titles  taken  from  the 
data.  Their  reactions  to  these  lists  inr -  ated  the  degree  to  which  the 
lints,  and  hence  the  original  data,  actualxy  represented  the  actions  and 
attitudi  s  which  were  important  to  them. 

2.  Construction  of  Lists 

A  "jiessage"  consisted  of  a  list  of  category  titles  from  one  classi¬ 
fication  session.  In  this  study,  the  titles  were  selected  on  the  basis 
of  category  frequency,  i.e.,  the  total  number  of  responses  in  each  category. 

The  or  ;inal  UCLA  positive  categories  were  ranked  bv  frequency  and 
two  lists  were  constructed  based  on  those  rankings:  Message  I  included 
the  fi  e  most  frequent  categories  (ranks  1-5)  and  Message  II  the  second 
five  most  frequent  categories  (ranks  6-10).  The  original  Stanford  posi¬ 
tive  < ategories  were  also  ranked  by  frequency  and  two  messages  (III  and 
IV)  constructed  in  a  similar  fashion.  Lists  were  also  based  on  the  live 
most  frequent-  negative  categories  from  both  the  Stanford  and  UCLA  subject 
populations . 

3 .  Presentation  of  Lists 

The  lists  were  presented  in  printed  test  booklets.  Each  page  of  a 
booklet  included  the  lists  to  be  com-,  iret  ,  the  question  stating  the 
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criteria  for  comparison,  and  an  answer  blank  in  which  the  code  number  of 
the  selected  list  could  be  written.  The  arrangement  of  the  lists  on 
each  page  was  varied  as  was  the  order  in  which  the  pages  were  arranged 
in  the  booklets.  Booklets  were  passed  out  to  Ss  in  random  order.  These 
controls  were  introduced  to  prevent  any  confounding  effects  which  might 
arise  from  the  order  in  which  the  lists  were  presented  and  fatigue  or 
boredom  which  might  occur  toward  the  end  of  the  session. 

A .  Procedure 

The  70  subjects  were  told  that  the  lists  were  written  by  their  peers 
in  an  attempt  to  describe  their  actual  and  idealized  self-images  (concepts 
known  to  the  class).  Half  of  the  subjects  were  instructed  to  compare 
lists  I  and  III  and  to  select  the  one  which  best  described  their  idealized 
self-images  and  to  compare  lists  II  and  IV  on  the  same  basis.  The  same 
two  pairings  of  lists  were  presented  to  the  other  half  with  instructions 
to  compare  them  on  the  basis  of  their  actual  self-images.  The  subjects 
were  also  presented  with  two  negative  messages  and  instructed  to  select 
the  one  which  they  and  their  peers  would  be  most  likely  to  disapprove. 
Finally,  individual  negative  category  titles  were  listed  and  the  subjects 
indicated  the  persons  (sources)  \o  would  be  most  displeased  if  they  did 
the  things  listed. 

■  Evaluation  of  Validity  Test  Session 

1.  The  validity  test  appeared  to  be  a  logical  method  for  evaluat¬ 
ing  the  effectiveness  of  the  data  collection  instrument.  The 
procedure  was  adequate  in  that  the  subjects  were  able  to  dis¬ 
criminate  between  the  various  lists  and  to  follow  the  iusti ac¬ 
tions  in  making  their  comparisons. 

2.  The  use  of  a  printed  test  booklet  was  an  efficient  method  for 
presenting  the  lists  to  be  compared  and  for  recording  the 
subjects'  responses. 
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3.  On  the  basis  of  the  success  of  the  procedure  and  the  inherent 
logic  of  the  validity  test  design,  similar  tests  were  employed 
at  Carnation  and  Prudential. 

G.  VALIDITY  TESTS — CARNATION  AND  PRUDENTIAL 

1.  List  Construction 

Three  methods  of  selecting  the  categories  to  be  included  in  the 
messages  were  tested  in  the  Carnation  and  Prudential  message  sessions 
which  were  held  in  the  same  week: 

1.  Categories  which  the  classifiers  ranked  as  most  important. 

2.  Categories  containing  the  responses  which  the  subjects 
ranked  as  most,  important. 

3.  Categories  that  had  the  highest  frequencies  (i.e.,  the 
greatest  number  of  responses  within  them.) 

In  addition,  "experts”,  persons  who  had  intimate  and  detailed 
knowledge  of  the  suoject  culture  but  were  not  part  of  it,  were  asked 
to  list  the  attitudes  which  they  felt  were  most  prevalent  within  and 
important  to  the  subject  population.  These  lists  were  similar  in  form 
to  the  ECHO-generated  messages  and  were  termed  "expert  messages."  The 
"experts"  employed  in  this  study  were  a  member  of  the  personnel  depart¬ 
ment  and  a  supervising  secretary. 

2 .  Criteria  for  List  Comparison 

The  standard  form  of  the  list  comparison  question  was  as  follows: 
"Which  of  the  messages  lists  the  actions  and  attitudes  which  you  feel 
are  most  important?" 

3.  Message  Session  Method 

a.  Subjects 

Ninety-eight  Prudential  employees  participated  in  the  first  message 
session.  In  the  Carnation  message  session,  15  Ss  were  secretaries  and 
16  held  non-secretarial  positions. 
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b.  Standard  Procedure 

The  messages  were  printed  on  s'  sets  which  were  organized  into  test 
booklets.  The  standard  sheet  had  printed  on  it  the  messages  to  be  com¬ 
pared  (two  or  four  messages),  the  question  stating  the  criteria  for  com¬ 
parison  when  different  questions  were  to  be  answered,  and  an  answer  blank. 
The  order  in  which  the  messages  were  arranged  on  any  one  sheet  was  varied 
to  control  for  possible  ordering  effects.  Likewise,  the  order  in  which 
the  sheets  were  arranged  in  the  test  booklet  was  varied  to  control  for 
effects  of  ordering,  fatigue  and  boredom. 

The  test  booklets  were  distributed  randomly  among  the  subjects  who 
were  instructed  to  read  the  messages  and  answer  the  question(s)  by  writ¬ 
ing  th(:  code  number  of  the  selected  message  in  the  answer  blank.  The 
subjects  were  permitted  to  work  at  their  own  speed  and  to  leave  the  room 
upon  completion  of  the  test. 

Special  test  booklets  were  prepared  for  different  subgroups.  For 
example,  the  Carnation  subjects  were  divided  into  two  groups,  secretaries 
and  non-secretaries,  and  different  messages,  based  on  secretarial  and 
non-secretarial  data  respectively,  were  prepared  for  them. 

c.  Variations  in  Message  Session  Procedure 

To  examine  the  effectiveness  of  the  ECHO  instrument  in  discovering 
the  power-structure,  or  hierarchy  of  sources  of  reinforcement,  the  message 
session  subjects  were  presented  with  individual  category  titles  and 
instructed  to  indicate  the  person  who  would  be  most  likely  to  approve  or 
disapprove  of  each  action  or  attitude.  The  resultant  distribution  of 
the  sources  of  approval  and  disapproval  attributed  to  a  specific  category 
was  later  compared  with  the  original  source  distribution. 

An  attempt  was  made  in  the  Carnation  and  Prudential  message  sessions 
to  test  the  efficacy  of  creating  a  hierarchy  of  values  on  the  basis  of 
category  frequencies  alone.  The  subjects  were  instructed  to  select 
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individual  category  titles  in  a  paired  comparison  format.  The  frequencies 
of  selection  were  compared  with  the  original  ranking  of  the  categories 
by  frequency. 

4.  Results  and  Evaluation 

a.  Statement  of  Criteria  for  List  Comparison 

One  result  of  the  Carnation  and  Prudential  validity  tests  was  the 
discovery  of  the  need  to  define  the  role  of  the  subject  and  to  clearly 
indicate  the  person  whose  opinion  is  being  sought. 

The  exact  wording  of  the  list  comparison  question  greatly  influenced 
the  manner  in  which  the  subjects  responded.  For  example,  when  asked  to 
select  the  "most  Important"  messtges,  the  Carnation  subjects  were  confused 
about  the  role  they  should  assume.  That  is,  one  message  might  be  more 
important  to  a  subject  in  terms  of  her  role  as  an  employee  while  another 
might  be  more  important  to  her  in  her  role  ns  a  mother.  It  was  necessary, 
therefore,  to  define  "importance"  in  terms  of  the  role  the  subject  was  to 
assume  in  making  her  selection. 

The  following  are  examples  of  ways  in  which  the  message  comparison 
question  was  stated: 

Which  of  the  messages  described  actions  and  attitudes  which 
you  feel  are  most  important  to  doing  your  job  in  the  Company? 

Though  this  question  was  effective  in  assigning  the  employee  role,  another 
problem  arose.  The  subjects  asked:  "Important  to  whom?  To  me?  To  my  boss?" 
It  was  necessary,  therefore,  to  indicate  not  only  the  role,  but  also  the 
person  whose  opinion  was  sought. 

Which  of  the  messages  describes  the  actions  and  attitudes  in 
doing  your  job  which  are  most  important  to  your  immediate 
superiors  (i.e.,  your  boss,  supervisor  and  manager)?  That 
is,  which  message  lists  the  actions  which  they  would  be  most 
likely  to  approve  of  your  doing? 
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This  question  not  only  defined  the  subject's  role  as  an  employee,  but 
it  also  identified  the  person  whose  opinion  of  importance  was  being 
sought. 

Which  message  lists  the  actions  and  attitudes  which  you 

believe  are  the  most  important  aspects  of  doing  your  job  and 

to  being  satisfied  and  personally  content  in  your  work? 

This  question  was  effective  in  assigning  the  subject's  role  and  in 
eliciting  his  personal  opinion. 

A  second  validity  test  was  held  at  Prudential  to  reevaluate  the 
new  message  selec  ion  criteria.  When  the  roles  were  clearly  defined, 
the  subjects  chose  the  ECHO-generated  frequency  messages.  These  findings 
were  applied  in  all  subsequent  sessions. 

b.  Evaluation  of  List  Construction 

(1)  Importance  Rankings 

The  subject's  rankings  of  their  responses  in  terms  of  importance 
were  found  to  be  ineffective  as  a  means  of  ranking  the  attitudes  held 
by  the  whole  group  since  the  individual  importance  ranks  were  randomly 
distributed  throughout  the  categories  (see  Appendix  II) . 

This  study,  while  not  conclusive,  indicated  that  the  importance 
rankings  assigned  to  the  categories  by  the  classifiers  were  not  represen¬ 
tative  of  the  hierarchy  of  attitudes  held  by  the  subject  population. 

This  conclusion  was  reached  when  the  subjects  regularly  rejected  the 
messages  created  on  the  basis  of  classifiers'  rankings. 

(2)  Category  Frequencies 

Selection  of  those  categories  containing  the  highest  frequencies 
appeared  to  be  the  most  effective  means  to  date  for  discovering  some 
prevalent  attitudes  within  the  value  system  of  the  subject  population. 
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H.  DEVELOPMENT  OF  CROSS-CLASSIFICATION  PROCEDURE— UCLA 

I.  Purpose 

The  cross-classification  or  triangulation  procedure  was  designed  to 
accomplish  two  goals:  (1)  To  objectively  redefine  the  meanings  of 
category  labels  created  by  the  classifiers;  (2)  to  quantify  differences 
in  social  perception. 

The  end  result  of  a  classification  session  was  a  system  of  category 
titles  which  defined  the  individual  responses  witnin  the  categories. 

These  titles  were  limited  as  they  represented  the  meanings  attributed  to 
the  responses  by  only  one  team  of  classifiers.  In  the  triangulation  pro¬ 
cedure,  at  least  three  different  teams  classified  the  same  data.  In  this 
manner,  the  meaning  of  a  category  title  was  redefined  in  terms  of  the 
category  labels  assigned  by  the  other  classification  teams  to  the  respon¬ 
ses  which  were  included  in  the  first  category.  These  different  labels 
represented  different  ways  of  verbalizing  the  content  of  the  responses 
and  served  to  make  the  description  of  those  responses  much  more  detailed 
than  the  category  title  resulting  from  one  classification  system. 

For  example,  the  M2  sample  was  classified  by  classification  teams 
M2,  M^  and  F^.  The  highest  ranking  category  (in  terms  of  frequency} 
created  in  the  M^  classification  was  Category  52,  "Be  a  good  student." 

The  category  labels  assigned  by  the  M2  £s  to  the  responses  in  M^'s  category 
52  Included:  "Get  good  grades  and  do  well  in  school",  "Be  accepted  and 
make  good  in  graduate  school",  "Improve  my  personal  and  physical  being", 

"Be  a  better  person  and  relate  to  my  environment  better",  and  "Find  a 
satisfying  occupation".  Likewise,  the  category  labels  assigned  to  the 
same  responses  by  the  F^  Cs  included:  "Do  better  In  school  because  of 
outside  pressure",  "Go  to  some  sort  of  graduate  school",  "Improve  sphere 
of  knowledge",  "Realize  minor  goals",  "Graduate  within  allotted  time", 

"Have  more  self-confidence",  and  "Ee  more  livable".  It  can  be  seen  that 
cross-classification  leads  to  a  more  detailed  and  specific  definition  of 
the  meaning  of  M^'s  category  label  "Be  a  good  student". 
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To  quantify  differences  in  social  perception,  the  compositions  of 
the  different  teams  classifying  the  data  were  varied.  For  example,  a 
team  of  men  and  a  team  of  women,  when  classifying  the  same  responses, 
created  different  categories  presumably  due  to  the  differences  in  their 
interpretations  of  the  data.  Cross-classification  is  a  means,  then,  for 
discovering  differences  in  social  perception  between  different  subcul¬ 
tures  or  groups  within  the  subject  population. 

2.  Data  Base 

The  data  samples  were  chosen  randomly  from  the  positive  responses 
made  by  the  subjects  in  the  UCLA  data  collection  session.  The  responses 
made  by  15,  15,  and  22  female  subjects  constituted  three  female  samples, 
F^,  F2,  and  ,  respectively.  The  three  samples  of  male  positive  respon¬ 
ses  (M^ ,  M2>  and  M^)  were  generated  by  15,  15,  and  JO  male  subjects, 
respectively. 

In  addition,  the  and  M2  samples  were  combined  as  were  the  and 
F2  samples.  These  combinations  resulted  in  two  "new"  samples  of  30 
subjects  each  which  were  also  classified. 

The  samples  containing  15  subjects  each  were  considered  sufficiently 
large  on  the  basis  of  the  Stanford  sample  size  experiments.  In  addition, 
each  of  these  samples  consisted  of  approximately  150  cards,  small  enough 
to  prevent  the  confounding  effects  of  fatigue. 

3.  Procedure 


a.  Composition  of  Classification  Teams 

Five  all-male  classification  teams,  each  one  composed  of  three 
classifiers  (teams  M^,  M2,  ,  and  M^)  and  four  all-female  teams  of 

three  classifiers  each  (teams  F^,  F2,  F^ ,  and  F^)  were  employed  in  the 
classification  of  the  samples  drawn  from  the  UCLA  positive  data.  Table 
10  indicates  which  samples  were  classified  by  each  of  the  classification 
teams . 
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TABLE  10 

CLASSIFICATION  OF  UCLA  SAMPLES 


An  "X"  indicates  that  the  team  (in  a  specific  row)  classified  che 
sample  (in  a  specific  column) . 

Classification  Sample 

Team  Mx  M2  M3  Fx  F2  F3  (ML  and  M2)  (Fx  and  F2) 


b.  Method 

Each  of  the  classification  teams  followed  the  standard  classifica¬ 
tion  procedure:  initial  (individual)  sorting  of  cards;  formation  of  teams 
and  creation  of  larger  categories;  refinement  of  categories  and  assignment 
of  labels.  The  £s  did  not  rank  the  categories. 

4.  Results 

a.  Time  Factors 

The  length  of  time  taken  to  complete  any  one  classification  is  affec¬ 
ted  by  the  size  of  the  task  (i.e.,  the  number  of  ca rA-  to  be  classified), 
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the  length  of  time  necessary  for  the  team  to  learn  the  classification 
procedure,  monotony,  boredom,  fatigue,  motivation,  and  skill. 

b.  Optimal  Conditions 

•  The  classification  sessions  should  not  exceed  three  hours  in 
length,  and  there  should  be  substantial  rest  periods  between 
sessions . 

•  Two  hundred  fifty  to  300  cards  is  the  optimal  number  of  cards 
that  can  be  handled  efficiently  by  one  team  of  three  classi¬ 
fiers  in  one  session. 

I .  CUBAN  STUDIES 

1.  Purpose 

The  application  of  the  ECHO  methodology  to  a  somewhat  alien  culture 
in  a  language  other  than  English  was  seen  as  a  logical  step  in  the  process 
of  developing  the  ECHO  instrument  for  eventual  use  with  a  completely  for¬ 
eign  culture. 

2.  Subjects 

Two  groups  of  Cuban  refugees  living  in  the  United  States  served  as 
subjects.  The  first  group,  CE1,  was  composed  of  11  women  and  14  men; 
the  second  group,  K,  included  10  men  and  9  women.  All  the  subjects,  whose 
ages  ranged  from  17  to  69  years,  were  literate  and  most  of  them  had  at 
least  a  high  school  education.  The  two  groups  will  be  considered  together, 
as  both  received  the  same  experimental  treatment  and  conditions. 

3.  Data  Collection  Session 


a.  Materials 

Data  collection  packets  were  in  standard  form;  all  the  materials 
were  printed  in  Spanish.  As  with  previous  groups,  the  presentation  order 
of  the  positive  and  negative  cards  was  varied.  The  subjects  received 
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biographical  Information  cards  which  requested  information  about  sex,  age, 
marital  status,  education,  occupation,  and  length  of  residence  in  the 
United  States. 

b.  Procedure 

A  man  Indigenous  to  the  subject  group  read  the  instructions  in  Spanish 
to  the  su  jects.  The  standard  data  collection  procedure  was  followed  in 
the  distribution  and  completion  of  the  packets. 

4.  Data  Classification  Session 

The  standard  classification  procedure  was  employed.  The  several 
variations  in  the  composition  of  classification  teams  and  the  samples 
of  data  classified  are  shown  in  T>le  11. 

5.  Validity  Teat 

a.  Construction  of  Lists 

Several  "Frequency"  messages  were  constructed  based  on  the  classifi¬ 
cation  systems  shown  in  Table  11.  In  addition,  the  "Frequency"  message 
from  the  Stanford  positive  data  was  included. 

b.  Procedure 

Pencil  and  paper  test  booklets  were  prepared,  each  page  containing 
two  lists  printed  in  Spanish  and  an  answer  blank.  The  subjects  were 
instructed  to  select  from  each  pair  of  lists  "the  one  which  most  exactly 
describes  the  things  which  you  believe  are  most  important  to  you".  The 
booklets  r  .ceived  by  12  male  subjects  contained  message  pairings  which 
differed  slightly  from  those  prepared  for  the  13  females. 

6.  Results  and  Implications 

a.  Data  Collection  Session 

The  Cuban  study  represented  the  first  application  of  tile  ECHO  metho¬ 
dology  with  persons  speaking  a  language  other  than  English.  After  the 
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CLASSIFICATION  OF  CUBAN  SAMPLES 


S3AJ3B83J  ajBH 

M  9  ISO 

83AHTB0d  3JBW 
M  9  130 

83AfqB8;»n  axBmaj 
M  9  130 

83AX3XBOd  aiBmsi 
M  S  130 

83AJ3B83N  SXBmaj 
M 

83AX3T80J  3 JBUI3  X 

83AI3B83N  3IBH 

S3AflT80d  3IBW 

M 

83AJ3B83N  <9  83AX'4TBOJ 

3XB1B3J  X30 

83AX3B83N  *  B3AX3X80a 
S»T®W  130 

83AT5B83N  3XBD3J 
130 

83AX3TBOa  3XBTB3J 
130 

83AX3B83M  3XBH 
130 

93AT3TB0a  3XBW 
130 
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data  collection  questions  had  been  translated  into  Spanish,  no  problems 
arose  from  the  use  of  a  foreign  language,  as  it  was  not  necessary  to 
cranslate  the  responses  or  category  titles  into  English.  All  statistical 
and  computer  manipulation  of  the  data  utilized  the  code  numbers  assigned 
to  the  categories,  and  in  this  way,  the  meaning  or  context  of  the  category 
labels  created  by  the  indigenous  classifiers  was  unaltered. 

b.  Classification  Session 

The  cross-classifications,  e.g.,  the  classification  of  male  responses 
by  female  classifiers,  repealed  attitudinal  differences  between  males  and 
females  in  the  suhject  culture.  Although  the  females  were  often  astounded 
by  and  unable  to  classify  some  of  the  male  respon^  s,  they  were  able  to 
accept  those  responses  as  normal  and  understandable  when  they  were  presen¬ 
ted  in  the  context  of  the  classification  system  created  by  the  males, 

i.e.,  they  were  given  the  cards,  v.  th  the  male  category  labels  attached, 

and  were  asked  if  the  classifications  were  appropriate. 

c.  Validity  Ten* 

As  a  result  of  the  validity  test,  some  evidence  was  found  regarding 
the  question  of  whether  positive  and  negative  responses  should  be  classi¬ 
fied  together  (i.e.,  mixed  prior  to  the  classification  session)  or  separ¬ 
ately.  The  Cuban  males  were  presented  with  two  lists:  Message  I,  male 
positive  data  classified  by  males,  and  Message  V,  combined  male  positive 
and  negative  data  classified  by  males.  Likewise,  the  female  subjects 
were  instructed  to  compare:  Message  III,  female  positive  data  classified 
by  females,  and  Message  VI,  combined  female  positive  and  negative  data 
classified  by  females.  In  selecting  the  most  important  list,  both  groups 
selected  the  list  based  upon  the  separate  classification  of  positive 
responses.  That  is,  they  rejected  the  lists  which  were  generated  from 
the  classification  of  combined  positive  and  negative  responses;  the 
results  were  significant  beyond  the  0.10  a  level. 
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J.  OPERATIONAL  EVALUATION  OF  ECHO  METHODOLOGY — NORTHWESTERN  UNIVERSITY 

AND  NURSES 

1 •  Purpose 

Twc  new  subject  populations  were  tested  as  an  operational  evaluation 
of  the  ECHO  methodology  which  had  been  developed  and  refined  through 
previous  applications. 

2.  Subjects 

T1  .  two  samples  consisted  of  68  students  enrolled  in  an  undergrad¬ 
uate  psy  :hology  class  at  Northwestern  University  and  52  ferale  nursing 
students.  Both  groups  of  subjects  were  predominantly  freshmen  and  sopho¬ 
mores  and  ranged  from  18  to  22  years  old. 

3 •  Data  Collection  Session 

a.  Materials 

Data  collection  packets  were  prepared,  each  consisting  of  10  IBM 
cards  printed  with  the  questions  "What  is  a  good  thing  to  do?  Who  would 
approve?"  and  10  cards  printed  with  the  questions  "What  is  a  bad  thing 
to  do?  Who  would  disapprove?"  A  biographical  information  card,  placed 
last  in  the  packet,  solicited  information  regarding  *-he  subject's  age, 
sex,  marital  status,  and  academic  status,  and  the  number  of  years  he  had 
attended  the  school  at  which  he  was  then  enrolled.  In  addition,  the 
subject  was  instructed  to  write  on  the  biographical  information  card  the 
person  whose  approval  and  disapproval  was  most  important  to  him. 


The  Northwestern  subjects  were  randomly  divided  into  two  equal 
groups  (Group  R  and  Group  S)  and  the  Nurses  into  two  groups  (Group  P  and 
Group  Q).  These  divisions  were  made  merely  to  decrease  the  sizes  of  the 
samples,  and,  hence,  the  number  of  responses  to  be  classified  b'-  each 
team. 
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h*  Procedure 

The  packets  were  distributed  among  the  subjects  who  were  instructed 
to  work  at  their  own  speed  in  answering  the  questions  printed  on  the  cards 
Importance  rankings  of  the  responses  were  omitted  as  they  were  found,  in 
the  Carnation  and  Prudential  studies,  to  be  random  and  unreliab’s  esti¬ 
mates  of  the  importance  hierarchy  of  attitudes. 

** •  Data  Classification 


a.  Classifiers 

teams  of  classifiers  were  chosen  randomly  from  the  Northwestern 
subject  population.  The  composition  of  the  teams  was  as  follows: 

Description 
Two  males,  one  female 
Two  males,  one  female 
One  male,  two  females 
Two  males,  one  female 
Two  males,  one  female 
Three  females 
Three  females 

El8ht  classi^ation  teams,  each  consisting  of  three  women,  were 
chosen  randomly  from  the  Nurses  subject  population. 

b.  Cross-Classification — Distribution  of  Snples 

The  data  collection  sessions  resulted  in  four  Northwestern  samples 
(R+  and  S+,  positive  responses,  and  R-  and  S-,  negative  responses^  and 
four  Nurses  samples  <P+  and  Q+,  positive  responses,  and  ?-  and  Q- ,  nega¬ 
tive  responses).  Tables  12  and  13  indicate  which  samples  were  classified 
by  each  of  the  classification  teams. 


Team  Number 
1 
2 

3 

4 

5 

6 

8  (sic) 
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TABLE  12 

CLASSIFICATION  OF  NORTHWESTERN  SAMPLES 

An  "X"  indicates  that  the  team  (in  a  specific  row)  classified  the 
sample  (in  a  specific  column).  "N"  equals  the  total  number  of  cards  in 
each  sample. 

Classification  Sample 

Team  Ri-(N  ■=  330)  R-(N  -  325)  S+(N  =  347)  S-(N  *=  340) 

1  XX 

2  XX 

3 

4 

5  X 

6  X 
8  (sic) 

c.  Procedure 

Standard  data  classification  procedure  was  employed  with  both 
groups.  However,  a  reliability  estimate  was  introduced  in  the  Nurses 
classification.  A  third  team  of  classifiers  was  given  the  cards  in  each 
sample  and  instructed  to  sort  them  into  the  categories  previously  created 
by  another  team. 

5.  Validity  TeBts 

a.  Construction  of  Lists 

Frequency  messages  (lists)  were  based  on  each  of  the  classification 
systems.  In  addition,  frequency  messages  were  constructed  from  Stanford, 
Cuban,  and  UCLA  data.  Finally,  two  sex-specific  messages  were  based  on 


X  X 

X  X 

X 
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TABLE  13 

CLASSIFICATION  OF  NURSES  SAMPLES 


An  "X"  indicates  that  the  team  (in  a  specific  row)  classified  the 
sample  (in  a  specific  column) . 

An  asterisk  *  indicates  the  sorting  of  data  into  prepared  categor¬ 
ies.  (Reliability  estimate) 

"N"  equals  the  total  number  of  cards  in  each  sample. 


Classification 

Team 


P+(N  =  249) 


Sample 

P-(N  =  257)  Of (N  -  250) 


Q- (N  =  259) 


1 

2 

3 

4 

5 

6 

7 

8 


the  separate  distributions  of  the  ".ale  and  female  responses  among  the 
categories  created  by  team  6. 

b.  Procedure 

The  test  booklets  consisted  of  various  combinations  of  the  messages, 
presented  two  per  page.  From  each  pair  of  lists,  the  S>s  were  instructed 
to  select  the  one  list  which,  in  their  opinion,  was  more  important  overall. 
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In  order  to  present  different  combinations  of  lists,  the  Northwest¬ 
ern  subjects  were  divided  into  three  male  and  three  female  groups  and 
different  booklets  were  prepared  for  them.  Likewise,  two  different  book¬ 
lets  were  prepared  for  the  Nurses  subjects.  The  booklets  were  distributed 
and  the  subjects  instructed  to  complete  them  at  their  own  speed. 

6.  Results 

The  resnlts  were  encouraging  and  interesting,  as  the  validity  test 
Ss  were  able  to  discriminate  between  lists  of  attitudes  on  the  basis  of 
the  sex  and  school  of  the  data  collection  subjects  from  whose  responses 
the  lists  were  generated. 

In  addition,  the  applications  of  the  ECHO  methodology  to  the  Nurses 
and  the  Northwestern  students  reconfirmed  the  logic  and  efficiency  of 
the  procedure  itself.  The  method  as  tested  was  the  culmination  of  the 
various  development  changes  which  resulted  from  previous  studies.  As  of 
this  writing,  additional  data  are  being  collected  from  these  populations. 

K.  ELEMENTARY  SCHOOL  CHILDREN 

Data  were  collected  from  two  groups  of  school  children  (3rd  and  6th 
grades)  to  answer  two  specific  questions.  First,  can  the  IBM  card  form 
of  the  ECHO  instrument  be  applied  to  children,  and  what  is  the  minimum 
age  for  Ss  using  this  form?  And  secondly,  can  the  ECHO  technique  detect 
and  describe  the  process  of  "value  internalization,"  i.e.,  the  socializa¬ 
tion  process  and  the  sources  of  reinforcement  which  determine  its  course? 
The  following  tentative  findings  are  indicated: 

1*  Third  Graders 

"Above  average"  school  children  (ages  8-8  1/2),  in  the  last  quarter 
of  the  third  grade  are  capable  of  handling  the  IBM  card  form  of  the  ECHO 
qu  stionnaire.  The  Ss  understood  the  questions,  were  easily  motivated, 
and  were  very  conscientious  in  completing  the  questionnaire.  Children, 
ir.  fact,  seem  to  be  more  motivated  than  adult  Ss— every  child  completed 


UNCLASSIFIED 


79 


UNCLASSIFIED 


all  20  questions;  adults  sometimes  fall  to  finish  the  20  questions.  A 
drawback,  however,  Is  that  third  graders  require  1  hour  to  1  hour  and  20 
minutes  to  finish  the  questionnaire,  while  the  average  adult  requires 
onlV  20  minutes  to  complete  20  questions  and  rank  the  cards  on  an  "impor¬ 
tance  hierarchy". 

Third  grade  Ss  who  were  identified  as  being  "average"  or  "below 
average"  by  the  teacher  were  not  able  to  handle  the  IBM  form  of  the  instru¬ 
ment  in  an  acceptable  time  period.  The.?e  same  Ss ,  however,  were  able  to 
answer  the  20  ECHO  questions  when  the  task  required  answers  to  be  listed 
on  an  8  1/2"  x  11"  piece  of  lined  paper.  These  findings  suggest  that  all 
"normal"  American  fourth  graders  could  easily  handle  the  IBM  card  instrument. 

2.  Sixth  Graders 

Sixth  graders  (ages  11-12)  were  tested  with  the  standard  20  question 
ECHO  instrument.  The  data  were  compared  with  data  collected  from  third 
graders  in  an  effort  to  identify  and  quantify  the  changes  in  values  which 
occur  during  the  3->ear  period  between  3rd  and  6th  grade.  This  experiment 
indicates : 

1.  The  ECHO  instrument  can  detect  shifts  in  the  prevalence  and 
intensity  of  values  in  a  culture. 

2.  The  method  can  also  identify  the  "power  structure",  i.e., 
sources  of  reinforcement,  which  shaped  these  changes.  In 
effect,  this  tells  the  researcher  what  (or  whe)  influenced 
each  Ss'  evaluation  of  the  person,  object,  or  concept  under 
study. 

3.  Changes  in  the  "power  structure"  can  also  be  used  to  detect 
the  "internalization"  process,  i.e.,  the  process  of  adopting 
the  beliefs,  norms,  and  values  of  one's  referent  culture. 
Techniques  for  representing  this  information  quantitatively 
and  graphically  have  also  been  developed. 
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4.  Taken  together,  findings  (1)  and  (2)  indicate  that  "develop¬ 
mental"  investigations  can  be  executed  by  the  ECHO  technique 
within  a  relatively  short  period  of  time,  i.e.,  it  is  not 
necessary  to  wait  for  time  to  pass  to  study  changes  in  values 
if  it  can  be  inferred  that  the  differences  between  two  age 
groups  of  children  in  the  same  culture  are  due  to  the  culture's 
socialization  process. 

L.  ILLITERATE  SUBJECTS 

A  pilot  study  designed  to  evaluate  the  feasibility  of  an  oral  inter¬ 
view  form  of  the  ECHO  questionnaire  is  currently  in  progress. 

1.  Subjects 

Mexican-Americans  living  in  the  "East  L.A."  area  of  Los  Angeles  acted 
as  subjects.  Each  subject  was  interviewed  by  two  researchers;  responses 
were  recorded  on  IBM  cards.  Use  of  this  particular  population  permits  the 
continued  evaluation  of  the  instrument's  ability  to  work  in  an  "alien 
culture"  and  at  the  same  time  provides  information  about  the  problems 
inherent  in  testing  illiterates. 

2.  Findings 

Potential  problems  in  applying  the  ECHO  technique  to  illiterates 
inc.-  ude : 

1.  Loss  of  S.  anonymity — this  is  particularly  confounding  in 

societies  where  extreme  ingratiation  is  a  cultural  trait. 

2.  Excessive  time  requirements — these  Ss  required  ora  to  two 
hours  to  answer  20  questions  verbally;  literate  Ss  can  answer 
20  written  questions  in  twenty  minutes. 

3.  A  method  of  classifying  the  data  which  utilizes  indigenous 
classifiers  must  be  developed. 

4.  Validity  tests  (message  tests)  for  illiterate  populations  must 
be  developed.  Tape  recorded  lists,  presented  with  rigid 
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experimental  controls  for  experimenter  bias,  presentation 
effect,  etc.,  are  under  consideration  as  a  possible  solution 
to  the  problem. 

Possible  solutions  to  these  and  other  problems  have  been  given 
some  consideration  during  Phase  I;  however,  empirical  evidence  has  not  yet 
been  collected. 
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APPENDIX  II 

CONSTRUCTION  OF  VALIDITY-TEST  MESSAGES 


The  format  of  the  validity  tests  requires  that  data  from  the  target 
population  and  data  from  several  other  populations  be  "echoed"  back  to  a 
sample  of  the  target  population  for  approval  or  disapproval.  The  hypothe¬ 
sis  is  that  subjects  will  select  the  data  that  came  from  their  population 
over  data  from  any  other  population.  The  data  in  this  case  are  category 
titles  which  were  generated  by  teams  of  classifiers.  The  classification 
process  produces  lists  of  category  descriptions,  frequencies  of  mention 
for  each  category,  and  classifiers'  importance  rankings  of  each  category. 
Other  information  is  available  within  each  category:  the  importance 
ranking  each  .S  gave  to  the  cards  be  submitted,  the  order  in  which  the 
cards  in  a  category  are  completed,  the  number  of  Ss  who  contributed  to 
the  category,  and  how  other  classifiers  sorted  the  same  cards. 

Problem.  What  portion  of  the  available  data  contains  the  most 
veridical  information  about  the  subject  population’s  value  system,  i.e., 

which  of  the  possible  samples  of  the  data  will  they  say  is  most  important 
to  them? 

Method.  Generate  lists  of  values  in  different  ways  and  see  which 

method  produces  preferred  lists.  Lists  were  produced  in  the  following 
ways: 

1*  Experts  submitted  lists  of  values  which  they  thought  the 

population  held.  The  first  five  items  were  included  in  the 
'Expert's"  list. 

2.  Classifiers  were  asked  to  rank  all  of  their  categories  on  an 
importance  scale.  The  five  highest-ranked  categories  were 
selected  for  the  "Importance  to  Classifiers"  list. 
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3.  The  five  categories  with  the  highest  frequency,  i.e.,  the 
categories  with  the  largest  number  of  cards  in  them,  were 
included  in  the  "Frequency"  list. 

4.  Each  subject  had  ranked  his  responses  on  an  importance  scale; 
by  taking  the  responses  ranked  "1"  it  was  hoped  that  a  weigh. - 
ing  for  importance  could  be  achieved.  The  five  categories 
with  the  largest  number  of  "1"  importance  values  were  included 
in  the  "Importance  to  Ss"  list. 

The  lists  were  presented  in  a  paired-comparison,  forced-choice  format  and 
jJs  ware  asked  to  select  the  list  which  best  reflected  their  views. 

Results.  The  "Frequency"  lists,  (3)  above,  proved  to  be  the  most 
popular  of  the  lists  in  several  different  validity  test  sessions. 

Discussion.  "Importance  to  :Ss"  rankings  and  categories  proved  to 
be  statistically  independent,  i.e.,  no  category  came  up  with  a  prepon¬ 
derance  of  the  higher  "importance"  ratings  (see  Table  14  for  example). 

By  taking  the  distribution  of  all  of  the  cards  ranked  "1"  we  were  merely 
sampling  10%  of  the  population  data  distribution.  The  Law  of  Large 
Numbers,  which  is  applicable  here,  tells  us  that  the  variance  increases 
as  the  sample  size  decreases.  By  taking  1/10  of  the  responses  we  merely 
increased  the  variance  in  the  frequency  distribution  from  which  we  selec¬ 
ted  the  five  largest  categories.  This  method  of  selecting  categories 
would  not  be  expected  to  produce  "preferred"  lists;  and,  in  fact,  it  did 
not. 


Another  interesting  characteristic  of  the  "importance  ranking"  was 
discovered  in  the  course  of  post-test  data  analysis — importance  rankings 
a.e  dependently  related  to  the  order  in  which  the  cards  were  filled  out. 
That  is,  the  cards  filled  out  first  were  usually  given  the  higher  (I) 
rankings.  Table  15  illustrates  the  dependency  finding  which  obtained 
in  all  data  checked. 
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TABLE  15 

COr  {ELATION  BETWEEN  ORDER  AND  IMPORTANCE  OF  ITEMS 
Response  Sequence  Number 


1 

2 

3 

4 

5 

6 

1 

8 

9 

10 

1 

18 

6 

12 

5 

9 

5 

1 

4 

4 

8 

72 

•; 

13 

18 

6 

4 

7 

3 

3 

9 

4 

4 

71 

3 

7 

16 

7 

11 

7 

4 

» 

8 

4 

*4 

70 

Chi  Square  =  180.961 

4 

5 

7 

11 

13 

8 

1 

5 

2 

6 

7 

71 

with  81  degrees  of 
freedom  is  significant 

5 

7 

6 

5 

8 

9 

8 

10 

8 

5 

4 

70 

at  the  a  =  .005+  levt  1 

6 

4 

4 

11 

8 

4 

11 

8 

1 

14 

4 

69 

Therefore,  reject  the 
null  hypothesis  of 

7 

6 

2 

5 

/ 

7 

7 

12 

10 

4 

7 

67 
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4 

5 

6 

9 

8 

13 

5 

8 

8 
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9 

7 

4 

4 

2 

7 

8 

8 

10 
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5 

66 

10 

2 

4 

4 

7 

4 

6 

5 

11 

8 

17 

68 

NA 

2 

2 

3 

2 

2 

3 

2 

2 

2 

2 

22 

73 

73 

73 

73 

73 

70 

69 

70 

70 

70 

714 

Classifier 

’s 

Rankings 

of 

Importance. 

The  eliaoility  with  which 

erent  teams 

of 

class 

if iers 

rank 

the 

same 

categories 

in  the  same  order 

of  importance  appears  to  be  high.  For  example,  the  correlation  between 
th  '  importance  rankings  of  tvo  different  teams  of  Company  C  classifiers 
was  found  to  be  +.79;  the  data  and  statistical  tests  whiv.n  support  this 
statement  are  contained  in  Table  10. 


The  relationship  between  categories  ranked  on  importance  by  the  Cs 
and  ranked  on  frequency  of  mention  is  not  so  striking,  as  the  data  in 
Table  17  illustrate.  The  failure  of  the  "Importance  to  Cs"  technique 
can  probably  be  attributed  to  Cs  variance;  the  "Frequency"  technique 
minimizes  (but  does  not  eliminate)  the  variance  between  Cs. 
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TABLE  16 

CLASSIFIERS'  IMPORTANCE  RANKINGS 
(Two  Groups  of  Secretaries  Classifying  Data 
from  the  Second  Group) 


Rank  Assigned 

Rank  Assigned  Difference, 

d. 

1 

by  First  Group 

by  Second 

Group  Between  Rank 

No 

1  s 

A  a 

12 

13 

1 

1 

.1 

20 

15 

5 

25 

B 

26 

11 

15 

225 

C 

10 

8 

2 

4 

D 

18 

22 

4 

16 

E 

24 

24 

0 

0 

F 

15 

19 

■4 

16 

G 

22 

16 

6 

36 

H 

5 

4 

1 

1 

I 

23 

18 

5 

25 

J 

9 

14 

5 

25 

K 

1 

5 

4 

16 

L 

19 

20 

1 

1 

M 

16 

21 

5 

25 

N 

29 

26 

3 

9 

0 

6 

6 

0 

0 

P 

2 

2 

0 

0 

Q 

11 

9 

2 

4 

R 

13 

12 

1 

1 

s 

25 

27 

2 

4 

T 

28 

28 

0 

0 

t! 

8 

7 

1 

1 

V 

4 

1 

3 

9 

W 

17 

25 

8 

64 

X 

21 

17 

4 

16 

Y 

27 

29 

2 

4 

Z 

3 

23 

20 

400 

BB 

30 

30 

0 

0 

DD 

14 

10 

4 

16 

CC 

7 

3 

4 

16 

N  -  30 

Id 

2  -  960 

Spearman's  Rho  (o) 

for 

correlation 

between  rankings: 

A  _  1  .. 

6Id2 

5760 

1  5760  .  1  91 

.79 

P  *  1  - 

N(N2-  1) 

I  “ 

(30) (899) 

1  26970  1  "21 

n 
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TABLE  17 

RELATIONSHIP  OF  CLASSIFIERS'  IMPORTANCE  RANKINGS 
OF  CATEGORIES  TO  CATEGORY  FREQUENCIES 


Cs 

Data 

Spearman  Rho 

Significance  of  Rho 

PI 

Prudential  + 

p  =  0.3b 

t 

=  1.724 
df  =  20 

N.S. 

P2 

Prudential  - 

p  =  0.03 

t 

=  0.156 
df  =  27 

N.S. 

P3 

Prudential  4 

p  =  0.83 

t 

=  6.308 
df  =  18 

4 

a  =  .0014 

CA 

Carnation  4 

p  =  0.42 

t 

=  2.36 
df  =  26 

4 

a  =  .054 

CB 

Carnation  4 

p  *  0.18 

t 

=  1.008 
df  =  28 

N.S. 

An  important  use  of  Cs  rankings  of  categories  will  be  as  a  check 
against  spurious  f  rankings.  Although  the  multiple  classification  method 
normally  reveals  small  categories  that  could  be  considered  to  be  the 
breakdowns  of  larger  categories,  Cs  rankings  could  be  used  as  a  check 
against  the  possibility  that  all  three  teams  used  the  same  category 
widths  bat  disagreed  about  the  classifications. 
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APPENDIX  III 

ELEMENTARY  SCHOOL  STUDIES 


The  experiments  reported  here  were  designed  to  answer  two  major 
questions:  (1)  How  old  and  literate  must  our  subjects  be  to  use  the 

standard  IBM  card  form  of  the  ECHO  questionnaire?  (2)  Can  the  ECHO 
technique  detect,  and  perhaps  quantify,  the  process  of  'value  internal¬ 
ization"  or  "socialization"? 

Subjects 

Group  A:  female  and  male  students  in  the  third  grade  at  a  public 

elementary  school  in  Santa  Ana  (Orange  County),  California. 

Group  B:  female  and  male  students  from  a  sixth  grade  at  the  same 
elementary  school. 

Method.  A  standard  twenty-card  instrument  containing  the  questions: 

"What  is  a  good/bad  thing  to  do?"  and  "Who  would  appnve/disapprove?"  was 
completed  by  each  S_.  The  order  of  presentation  (good/bad  or  bad/good) 
was  reversed  for  half  of  each  sample  to  i  ontrol  for  sequential  effect. 

Standard  instructions,  modified  for  the  younger  populations,  were 
read  to  both  groups. 

A.  AGE  AND  LITERACY  REQUIRED  TO  USE  THE  ECHO  INSTRUMENT 

Problem.  How  old  and  literate  must  subjects  (Ss)  be  to  use  the  standard 
IBM  card  form  of  the  ECHO  questionnaire?  Can  children  comprehend  the 
task?  Do  children  generate  data  that  are  meaningful? 

1  2 

Bavelas  and  Kalhorn  found  that  fourth  graders  (ages  9  to  10) 
were  the  youngest  subjects  capable  of  handling  a  written  form  of  the 
questionnaire.  Third  graders  were  excluded  from  Bavelas'  studies  because 
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their  handwriting  was  illegible.  All  of  his  subjects,  including  pre¬ 
schoolers  who  were  given  oral  interviews,  were  able  to  comprehend  the 
task  and  generate  the  requested  responses. 

Findings.  Second-semester  third  grade  students  whom  the  teacher  rated 
as  "average  and  above"  were  able  to  handle  tue  standard  ECHO  instrument; 
"below  average"  students,  however,  could  not  satisfactorily  handle  the 
IBM  cards  even  though  they  could  give  acceptable  responses  when  given  an 
ordinary  piece  of  writing  paper. 

These  third  grade  students  required  one  and  a  half  hours  to  com¬ 
plete  twenty  cards  as  compared  to  an  average  adult  time  of  20  minutes. 

The  sir^h  graders  completed  the  task  in  approximately  the  same  time  as 
adults. 

Conclusion.  Children  with  a  fourth  grade  education  or  more  are  capable 
of  handling  the  standard  ECHO  instrument.  Third  graders  can  answer  the 
questions  but  they  require  a  long  time  to  complete  the  task.  This  find¬ 
ing  will  vary  from  culture  to  culture,  but  it  suggests  that  the  current 
ECHO  methodology  can  be  used  at  the  lower  levels  of  education. 

B.  MEASUREMENT  OF  "SOCIALIZATION" 

Problem.  As  children  mature,  they  appear  to  internalize  the  values 
of  the  subculture  in  which  they  live.  Adults  commonly  have  sets  of  values 
which  cause  them  to  behave  in  particular  ways  because  they  vxew  the  be¬ 
havior  as  being  "right"  or  "wrong".  No  outside  agency  is  required  to 
reward  or  punish  such  behavior.  An  assumption  is  that  the  behavior  and 
the  values  attached  to  it  were  learned  in  such  a  way  that  the  enforcing 
agency  is  transferred  from  the  outside  (e.g.,  from  "mother")  to  the  inside 
("self").  The  internalization  process  presumably  occurs  over  time.  The 
question  of  when  that  occurs  might  be  answered  by  determining  the  differ¬ 
ence  in  responses  to  the  question  'Who  would  approve/disapprove?"  for 
various  behaviors  by  children  of  different  age  groups. 
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Hypothesis .  Children  in  the  sixth  grade,  having  progressed  farther 
in  the  socialization  process,  will  respond  to  the  question  "Who  would 
approve/disapprove?"  with  the  concept  "self"  significantly  more  often 
than  children  in  the  third  grade;  the  converse  is  true  for  the  concept 
"parent  (=■)" . 

Results.  The  number  and  percentage  of  "myself"  responses  to  "Who 
woul'1  approve/disapprove?"  for  each  group  is  shown  in  Table  18. 

TABLE  18 

"MYSELF"  AS  A  SOURCE  OF  APPROVAI  OR  DISAPPROVAL 
IN  TWO  POPULATIONS  OF  ELEMENTARY  SCHOOL  CHILDREN 

%  Time  "Myself"  was  Source 


Good  Thing 

Bad  Thing 

Mixed 

Male  Third  Graders 

8.3 

13.3 

10.8 

Female  Third  Graders 

20.1 

20.0 

20.4 

Male  Sixth  Graders 

29.4 

19.3 

24.7 

Female  Sixth  Graders 

21.3 

19.0 

20.1 

Figure  6  includes  the  data  from  Groups  A  and  B  and  from  an  adult 
group.  The  curves  indicate  clearly  that  the  socialization  process  moves 
sharply  in  the  period  between  the  third  and  sixth  grades. 
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Figure  6.  "Socialization":  The  Internalization  of  Value  Systems 


UNCLASSIFIED 


APPENDIX  IV 

DETAILED  DESCRIPTION  OF  METHOD 


The  following  detailed  description  of  the  basic  ECHO  methodology 
as  of  June  1167  is  a  consolidation  of  material  appearing  in  earlier 
sections  of  this  report.  It  is  presented  in  this  form  for  convenience. 
It  includes  only  one  form  of  the  projective  question;  additions  and 
changes  are  expected. 

A.  DATA  COLLECTION  SESSION 

1.  Data  Collection  Packets 


a.  Data  Collection  Cards 

Data  collection  packets  were  prepared  prior  to  the  session,  each 
packet  consisting  of  20  IBM  cards  which  were  prepunched  with  the  follow¬ 
ing  information: 

1.  A  number  identifying  the  population  from  which  the 
sample  was  drawn 

2.  A  subject  number 

3.  A  sequence  number  to  indicate  the  order  in  which  the 
subject  wrote  his  responses 

4.  A  plus  ("+")  or  minus  to  indicate  whether  the 

question  on  the  card  was  ‘'positive"  or  "negative". 

Of  the  20  data  collection  cards  per  packet,  10  had  printed  on  them  the 
following  "positive"  questions:  "What  is  a  good  thing  to  do?"  and  "Who 
would  approve?"  Likewise,  the  10  "negative"  cards  had  the  questions: 
"What  is  a  bad  thing  to  do?"  and  "Who  would  disapprove?" 

A  sample  data  collection  card  is  shown  in  Fig.  7. 
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Figure  7.  Sample  Data  Collection  Card 


b.  Sequential  Controls 

To  control  for  the  possible  confounding  effects  of  the  order  in 
which  the  positive  and  negative  cards  were  presented,  half  of  the  sub¬ 
jects  in  each  sample  received  10  positive  cards  first  and  then  10  nega¬ 
tive  cards;  vice  versa  for  the  other  subjects. 

c.  Role  Instructions 

When  subgroups  within  a  sample  were  to  be  assigned  a  specific  role, 
role  instruction  cards  were  placed  first  in  the  packets  so  that  the  sub¬ 
jects  would  encounter  them  before  they  responded  to  the  data  collection 
cards.  For  example,  120  of  the  UCLA  Ss  were  instructed  tr  'spond  to  the 
questions  in  terms  of  their  roles  as  students  at  UCLA  while  the  ot.ier  30 
S_s  were  instructed  to  respond  in  terms  of  the  general  role  as  person  in 
society. 
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d.  biographical  Data  Cards 

The  final  card  in  each  data  collection  packet  was  the  Biographical 
Information  card  with  questions  appropriate  to  the  population.  The  Car¬ 
nation  and  Prudential  subjects,  for  example,  were  asked  about  sex,  educa¬ 
tional  background,  age,  job  title,  and  length  of  employment  with  the 
company.  The  biographical  data  card  employed  with  the  UCLA  and  Stanford 
samples  asked  about  the  subject's  age,  sex,  marital  and  academic  status, 
and  the  length  of  time  he  had  attended  that  school. 

2.  Data  Collection  Methodology 

The  data  collection  packets  were  distributed  randomly  to  the  sub¬ 
jects.  After  the  instructions  were  read  to  them,  the  subjects  were 
instructed  to  open  their  packets  and  begin.  They  were  allowed  to  pro¬ 
ceed  at  their  own  speed  and  were  asked,  upon  completion,  to  put  all  the 
materials  back  into  the  packets  and  to  return  the  packets  to  the  experi¬ 
menters. 


The  instructions  to  the  subjects  were  basically  the  same  for  all 
the  sample  studied.  The  format  was  as  follows: 


INSTRUCTIONS  TO  THE  SUBJECTS 


The  purpose  of  this  study  is  to  develop  methods  for 
improving  communications  between  different  cultures.  Any 
such  interculture  communication  is  necessarily  based  on  an 
understanding  of  the  value  systems  inherent  in  each  culture. 
We  are  currently  investigating  an  instrument  with  which  we 
hope  we  will  be  able  to  discover  and  examine  the  value  sys¬ 
tem  for  any  culture.  We  plan  to  test  the  effectiveness  of 
our  methodology  by  comparing  the  culture  which  has  developed 
in  this  group  with  a  somewhat  similar  culture  which  has 
developed  within  another  group  of  secretaries  [or  students]. 

Each  of  you  should  have  a  sealed  envelope  and  a  pen. 
These  are  the  only  materials  you  will  need.  You  will  be 
asked  to  give  ten  responses  to  each  of  the  following  ques¬ 
tions:  What  do  you  think  a  person,  like  yourself,  could  do 
that  would  be  a  good  thing  to  do  and  that  someone  would 
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approve  of  your  doing?  Who  would  approve  of  your  action? 
Likewise,  what  do  you  think  a  person,  like  yourself,  could 
do  that  would  be  a  bad  thing  to  do  and  that  someone  would 
disapprove  of  your  doing?  Who  would  disapprove? 

When  we  tell  you  to  begin,  you  are  to  open  the  en¬ 
velope  by  tapping  it  down  on  the  left  side  so  that  the 
cards  within  are  down  on  the  left.  Then  tear  off  the 
right  edge.  Take  out  the  deck  of  cards  which  are  num¬ 
bered  with  a  "1".  Do  not  remove  anything  else  from  the 
envelope.  Take  the  rubber  band  off  the  cards  but  be 
especially  careful  that  you  do  not  change  the  order  of 
the  cards.  Writing  or  printing  as  neatly  as  possible 
give  a  specific  example  of  a  good  or  a  bad  tiling  to  do, 
according  to  the  question  which  is  printed  on  the  card. 
Write  that  answer  on  the  lines  which  have  been  provided 
following  the  question.  Then  write  the  title  or  position 
of  the  person  who  would  approve  or  disapprove  of  your 
action  on  the  lines  which  have  been  provided  following 
that  question.  When  you  have  answered  all  ten  cards  in 
deck  1,  put  them  back  into  the  envelope  and  take  out  deck 
"2".  Again,  we  ask  you  not  to  change  the  order  of  the 
cards.  Follow  the  same  procedure  as  before  in  writing 
your  answers  on  these  cards.  When  you  are  finished,  re¬ 
turn  these  cards  to  the  envelope  and  take  out  the  yellow 
"Biographical  Information"  card.  Answer  each  of  the  ques¬ 
tions  on  this  card  by  placing  an  "x"  in  the  appropriate 
box.  Again,  when  you  are  finished  return  the  card  to  the 
envelope  and  all  your  materials  to  one  of  us.  At  that 
time  you  may  leave.  We  wish  to  thank  you  all  again  for 
donating  your  time  to  help  make  this  study  a  success. 

Do  you  have  questions?  If  a  problem  should  arise 
while  you  are  working,  please  raise  your  hand.  You  may 
begin. 


B.  CLASSIFICATION  SESSION 

The  classification  session  was  devised  as  a  method  for  reducing 
the  vast  amount  of  data  (20  separate  responses  ^er  subject)  into  a 
usable  and  logical  form. 


Persons  indigenous  to  the  subject  population  interpret  the  individ¬ 
ual  responses  made  by  the  subjects  and  put  together  those  responses  which 
are  similar  in  content  and  meaning.  It  is  essential  that  indigenous  clas¬ 
sifiers  be  employed  in  this  task,  as  persons  alien  to  the  subject  population 
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and,  hence,  alien  to  that  group's  valu<_  system,  would  make  different 
interpretations  of  the  data.  The  goal  of  the  classification  procedure 
is  to  reduce  the  masses  of  individual  responses  into  a  set  of  statements 
(categories)  which  describe  the  important  attitudes  held  by  the  subject 
population. 

1  •  Determination  of  Number  of  Classification  Teams  and  Length 

of  Session 

Teams  of  three  persons  each  were  chosen  randomly  from  the  subject 
population  to  serve  as  classifiers.  The  number  of  teams  depended  upon 
the  total  number  of  cards  to  be  classified.  Through  trial  and  error,  it 
was  determined  that  250-300  was  the  optimum  number  of  cards  that  could 
be  handled  by  a  group  of  three  classifiers.  In  cases  where  a  group  of 
classifiers  worked  with  more  than  300  cards,  boredom  and  fatigue  developed 
with  the  result  that  the  classifiers  became  less  critical  in  their  dis¬ 
criminations  and  tended  to  hurry  through  the  classification  task.  For 
example,  teams  from  UCLA  were  given  over  800  cards  to  sort.  The  process 
took  approximately  twelve  hours  and  the  resultant  categories  appeared  to 
the  investigator  to  be  rather  unreliable.  Whenever  large  numbers  of  cards 
are  to  be  processed,  the  classification  should  be  spread  over  a  larger 
period  of  time  by  having  work  sessions  of  no  longer  than  three  hours, 
with  a  substantial  break  between  sessions. 

2.  Procedure 

In  the  first  part  of  the  classification  session,  the  three  classi¬ 
fiers  worked  independently.  Each  was  given  about  100  cards,  i.e.,  one- 
third  of  the  deck  of  cards  assigned  to  his  team,  and  was  instructed  to 
stack  together  all  the  cards  which,  in  his  opinion,  said  or  meant  the 
same  thing. 

The  resultant  stacks  of  cards,  then,  represented  one  category  or 
type  of  response.  This  preliminary  sort  ng  task  took  roughly  one  hour 
to  complete. 
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In  •'he  second  part  of  the  session,  each  team  of  classifiers  was 
brought  together.  The  different  teams  were  physically  separated  from 
each  other  so  that  one  group  would  not  disturb  another.  The  experimen¬ 
ters  circulated  among  the  teams  of  classifiers,  being  care  ul  not  to  make 
any  suggestions  which  night  bias  the  classifiers'  decisions.  The  experi¬ 
menters  were  there,  for  example,  to  make  sure  that  no  one  member  dominated 
the  team's  discussions  and  to  expedite  the  process. 

This  phase  of  the  session  began  with  one  e^am  member  choosing  one 
of  his  stacks  and  reading  aloud  some  of  the  cards  it  contained.  The 
other  two  classifie-s  then  added  to  that  stack  any  of  their  cards  which 
they  felt  had  t>>e  same  meaning  as  those  being  read.  The  classifiers  took 
turns  reading  their  cards  aloud.  At  the  end  of  this  phase,  all  of  the 
classifiers'  separate  stacks  of  cards  had  been  combined  to  form  larger 
CCx  tegories  of  response. 

In  the  final  phase,  the  classifiers  were  instructed  to  reexamine 
the  newly-created  categories  and,  where  necessary,  to  break  them  down 
into  finer,  more  exclusive  categories.  They  then  labeled  each  category, 
giving  it  title  which,  in  the  form  of  a  response  to  the  original  ques¬ 
tion,  would  describe  all  of  tne  responses  within  the  category.  In 
addition,  the  Carnation  and  Prudential  classifiers  were  instructed  to 
rank  the  resultant  categories  from  1  to  10  in  terms  of  importance.  This 
ranking  was  later  omitted  from  the  classification  procedure  but  should 
be  retained  as  a  control  ove>-  eccentric  classifications. 

Typically,  the  classification  session  lasted  about  three  hours  and 
resulted  in  sets  of  20-10  categories  of  response. 

3.  Reliability  Estimates 

As  one  method  for  checking  the  reliability  of  a  team's  classifica¬ 
tion  system,  a  second  team  of  three  classifiers  was  given  the  list  of 
categories  created  by  the  first  team  and  instructed  to  re-sort  the  cards 
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intu  those  categories.  The  degree  to  which  the  cards  were  sorted  into  the 
sai'e  categories  by  the  two  different  teams  determined  the  degree  of  relia¬ 
bility  of  the  first  team’s  classification. 

Another  estimate  of  reliability  consisted  in  having  a  second  team 
of  classifiers  indicate  their  agreement  or  lack  of  agreement  with  the 
categories  created  by  the  first  team.  That  is,  the  members  of  the  second 
team  were  given  each  category  (stack  of  cards)  cr<  Ued  by  the  first  team 
and  asked  whether  or  not  the  cards  did  in  fact  belong  together  and  whether 
or  not  the  category  title  created  by  the  first  group  was  representative 
of  the  responses  in  that  category. 


4*  Cross-Classification  Systems 

Several  variations  in  the  types  of  data  classified  and  the  composi¬ 
tion  of  the  classification  teams  are  possible.  The  variables  include: 

!•  Data — sex  variable 

a.  Male  subjects’  responses  only 

b.  Female  subjects’  responses  only 

c.  Male  and  female  subjects’  responses  mixed 

2.  Data — age  variable 

a.  Responses  from  subjects  representing  different  age 

groups  can  be  classified  together  or  separately 

3.  Data  positive  and  negative  responses 

a.  Responses  to  the  "positive1’  and  "negative"  questions 
can  be  classified  together  or  separately 

4.  Classif lers--sex  variable 

a.  All  males 

b.  All  females 

c.  Males  and  females 
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5.  Classifiers — age  variable 

a.  Teams  of  classifiers  can  consist  of  only  persons  of 
a  specific  age  group  or  of  persons  of  different  ages 

6.  Classifiers — indigenous  or  alien 

a.  Although  indigenous  classifiers  are  essential  to  the 

original  interpretation  of  the  data,  it  would  be  inter¬ 
esting  to  discover  how  persons  alien  to  the  subject 
population  would  view  the  data.  In  this  manner,  dif¬ 
ferences  between  the  value  systems  of  the  subject 
population  and  the  dien  culture  might  be  discovered. 

Various  combinations  of  these  variables  are  po  ;sible  and  several 
have  been  used  to  date.  These  cross-classifications  reveal  important 
differences  between  subgroups  within  the  subject  population. 

5.  Classification  of  Sources 

The  power  structure  of  the  subject  population  is  represenced  by 
the  sources  of  approval  and  disapproval  of  tie  action--  listed  by  the 
subjects.  The  sources  can  be  grouped  or  classified  in  various  way', 
e.g.,  in  terms  of  their  relationships  to  the  subject^  or  their  status  in 
the  organization.  Indigenous  classifiers  are  needed,  particularly  with 
foreign  populations,  to  determine  the  hierarchy  of  the  sources  and  for 
other  groupings  of  sources. 

C.  MESSAGE  SESSIONS 

The  message  session  was  designed  tc  test  the  sensitivity  of  the 
ECHO  method;  to  determine  whether  the  subject's  responses  to  the  ECHO 
questions  do  in  fact  represent  attitudes  which  are  .important  and  preva¬ 
lent  in  the  value  system  of  the  subject  culture.  Lists  of  the  category 
labels  which  represent  such  attitudes  were  call°d  "messages".  Basically 
the  message  session  format  was  as  follows:  A  new  group  oi  subjects  was 
chosen  from  the  subject  population.  These  subjects  were  asked  to  compar 
various  messages,  e.g.,  ECHO  versus  expert;  ECHO  positive  versus  ECHO 
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negative,  and  to  select  the  message  which  listed  the  actions  and  attitudes 
they  considered  to  be  most  important  on  the  basis  of  certain  pre-stated 
criteria,  e.g.,  in  terms  of  their  roles  as  students  or  secretaries, 
dence,  the  message  session  tested  the  effectiveness  of  the  ECHO  instru¬ 
ment  in  discovering  prevalent  attitudes  within  the  subject  population 
and  also  served  as  an  examination  of  ECHO's  ability  to  discriminate  be¬ 
tween  different  subgroups  within  the  subject  culture. 

Message  Construction 
3.  ECHO  Frequency  Messages 

The  categories  within  each  classification  system  were  ranked  in 
terms  of  their  frequencies,  i.e.,  the  total  number  of  responses  in  each 
category.  The  categories  which  were  highest  in  rank  were  included  in 
the  message  and  their  labels  were  listed  in  order  of  decreasing  rank. 

different  frequency  messages  were  constructed  based  on  the  various  clas¬ 
sification  systems. 

b.  Expert  Messages 

Experts,  persons  who  had  intimate  and  detailed  knowledge  of  the 
subject  culture  but  were  not  part  of  it,  were  asked  to  list  the  atti¬ 
tudes  which  they  felt  were  most  prevalent  within  or  important  to  the 
subject  population.  These  lists  were  similar  in  form  to  the  ECHO- 
generated  messages. 


2*  Criteria  for  Message  Comparison 

As  the  message  session  was  designed  to  be  a  test  of  the  data  col¬ 
lection  method,  it  was  necessary  that  the  question,  i.e.,  the  statement 
of  the  criteria  on  which  the  subjects  were  to  base  their  comparisons  of 
the  messages,  parallel  the  data  collection  question  to  which  the  original 
responses  were  made.  The  standard  form  of  the  question  was  as  follows: 
Which  of  the  messages  lists  the  actions  and  attitudes  which  you  feel 
are  most  important?"  However,  it  was  discovered  early  in  the  study  that 
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the  exact  wording  of  that  question  greatly  influenced  the  manner  in 
which  the  subjects  responded.  Hence,  it  was  necessary  to  assign  a 
specific  role,  e.g.,  as  a  student  or  employee,  to  define  the  context 
in  which  the  subjects  were  to  make  their  comparisons  of  the  messages. 

In  addition,  the  question  itself  must  clearly  identify  the  person  whose 
opinion  is  being  sought.  The  question:  "Wh'ch  message  lists  the  actions 
and  attitudes  which  you  believe  are  the  most  important  aspects  of  doing 
your  lob  and  being  satisfied  and  personally  content  in  your  work?"  was 
effective  in  assigning  the  subject's  role  and  in  eliciting  his  personal 
opinion  rather  than  that  of  his  supervisor  or  co-workers. 

3.  Message  Session  Method 

a.  Standard  Procedure 

The  messages  were  printed  on  sheets  which  were  organized  into  test 
booklets.  The  standard  sheet  had  printed  on  it  the  messages  to  be  com¬ 
pared  (two  or  four  messages) ,  the  question  stating  the  criteria  for  com¬ 
parison  when  different  questions  were  to  be  answered,  and  space  for  the 
answer.  The  order  in  which  tiie  messages  were  arranged  on  any  one  sheet 
was  randomized  to  control  for  any  possible  ordering  effects.  Likewise, 
the  order  in  which  tiie  sheets  were  arranged  in  the  test  booklet  was  ran¬ 
domized  to  control  for  effects  of  ordering,  fatigue,  and  boredom. 

The  test  booklets  were  distributed  randomly  among  the  subjects 
who  were  instructed  to  read  the  messages  and  ai.  wer  the  question(s)  per¬ 
taining  to  then  by  writing  the  code  number  of  the  selected  message  in 
the  answer  blank.  The  subjects  were  permitted  to  work  at  their  own 
speed  and  to  leave  the  room  upon  completion  of  the  test. 

In  some  cases,  special  test  booklets  were  prepared  for  different 
subgroups.  For  example,  the  Carnation  subjects  were  divided  into  two 
groups,  secretaries  and  non-secretaries,  and  different  messages,  based 
on  secretarial  and  non-secretarial  data  respectively,  were  prepared  for 
them.  Similarly,  the  male  subjects  in  the  Cuban  and  Northwestern  groups 
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received  test  booklets  containing  messages  based  mainly  on  the  male 
responses  while  the  females  in  those  two  groups  received  messages  cre¬ 
ated  from  female  responses. 

b.  Variations  in  Message  Session  Procedure 

To  examine  the  effectiveness  of  the  ECHO  instrument  in  discovering 
ttic  power  structure,  or  hierarchy  of  sources  of  reinforcement,  the  mes¬ 
sage  session  subjects  were  presented  with  individual  category  titles  and 
instructed  to  indicate  the  person  who  would  be  most  likely  to  approve  or 
disapprove  of  each  action  or  attitude.  The  resultant  distribution  of 
tne  sources  of  approval  and  disapproval  attributed  to  a  specific  category 
was  later  compared  witli  the  original  source  distribution,  i.e.,  the 
sources  attributed  to  the  same  category  by  the  data  collection  subjects. 

An  attempt  was  made  in  the  Carnation  and  Prudential  message  sessions 
to  test  the  efficacy  of  creating  a  hierarchy  of  attitudes  on  the  basis  of 
category  frequencies  alone.  The  subjects  were  instructed  to  make  Daired 
comparisons  between  individual  category  title  and  the  frequencies  with 
which  the  categories  were  selected  were  later  compared  with  original 
ranking  of  the  categories  by  fre'uency. 


UNCLASSIFIED 


103 


UNCLASSIFIED 


THIS  PAGE  INTENTIONALLY  BLANK 


104 


UNCLASSIFIED 


UNCLASSIFIED 


REFERENCES 


Alex  Bavelas,  "A  Method  for  Investigating  Individual  and  Group 
Ideology,"  Sociometry,  V  (1942),  pp.  371-377. 

Joan  Kalhorn,  Kurt  Lewin,  et  al. ,  Authority  and  Frustration; 
"Studies  in  Topological  and  Vector  Psychology  III,"  University  of 
Iowa  Studies,  Studies  in  Child  Welfare  Vol.  XX,  University  of  Iowa 
Press  (1944). 

W.  L.  Warner,  et  al. ,  Democracy  in  Jonesville.  Harper  and  Brothers, 
New  York,  1949. 

Robert  J.  Havinghurst  and  B.  L.  Neugarten,  American  Indian  and 
White  Children,  a  Sociopsychological  Investigation.  University  of 
Chicago  Press,  1953. 

George  Hall  Rice,  "The  Interiorization  of  Attitudes  Among  Nurses 
at  the  Palo  Alto-Stanford  Hospital,"  Unpublished  Doctoral  Thesis, 
Stanford  University,  1965. 


UNCLASSIFIED 


105 


UNCLASSIFIED 


THIS  PAGE  INTENTIONALLY  BLANK 


1J6 


UNCLASSIFIED 


Security  Classification 


DOCUMENT  CONTROL  DATA  •  R&D 

(Security  ctaftti  cation  ot  ittia,  body  o/  mbit  fat  and  indexing  anno  tat  ion  mutt  bo  antarad  t^ifi  the  ovaratt  ft  port  i*  cla#** 


1.  ORIG1NATIN  C  ACTIVITY  (Corpofta  author)  2e.  REPORT  GECURiTy  c  L AttiyriCATiON 


Central  Research  Corporation,  Ine. 


J.  REPORT  TITLE 


2 b  group 


PROJECT  ECHO  ^  j 


4.  DESCRIPTIVE  NOTES  (Trpa  ol  rrport  mnd  Inebtln  dmtmm) 

Final  Report 


S  AU':HO«rS;  fL«#r  nant.  ftrsf  nnme,  Jnirto/j 

Barthol  R.  P. 

0.  Bridge 


6.  REPORT  DATE 

30  June  1967 


•  *.  CONTRACT  OR  GRANT  NO. 

DAHC04  67  C  0040 

a  PROJECT  NO. 

ARPA  ORDER  SO  965 


74-  TOTAL  NO.  OW  PAGEl  7*.  NO.  OP  REPt 

106  5 


Sa.  ORIGINATOR'S  REPORT  NUMRERfSj 

CR-0040-1 


9  b.  OTMjlR^RJROR  r  NO (S)  (Any  athar  numbaf  that  may  ba  aaat+tad 


to.  AVAIL  ABILITY/LIMITATION  NOTICES 

1 _ i*yy*-"* _ 

Distribution  of  this  report  is 

unlimited. 

It.  SUPPLEMENTARY  NOTES 

t*.  SPONSORING  MILITARY  ACTIVITY 

U.S.  Army  Research  Office -Durham 

None 

Box  CM,  Duke  Station 

'■w..,.,-.  k  r«W  n  «VB  m  m*TzWKm£it(  I 

13-  ABSTRACT 


The  ECHO  technique  utilises  a  "protective  surrey"  foraat*  the 
method ^projective  because  the  subjects  (8s)  art  permitted  to  define  sons 
variables  which  are  usually  predetermined  by  the  invest  1  gatorsy  bA% 
survey  because  groups  rather  than  Individuals  art  the  object  or  concern. 
The  value  of  this  type  of  attitude  survey  end  the  theoretical  logic  under¬ 
lying  its  development  are  explicated^  been. 


14.  KEY  WORDS 


projective  survey 

subcultures  (within  a  population) 

value  systems  (of  these  cultures) 


PORM 

1  JAN  14 


1473 


Security  ClftMifiotion 


